Background

Worldwide, it is estimated that 65 million people need manual or power wheelchairs (WCs) [1]. Older adults are four times more likely to rely on WCs [2], and their mobility needs are more complex due to a constellation of chronic conditions and comorbidities in comparison to younger adults [3, 4]. For example, after 50 years of age the sudden onset of disability presents a higher risk for restriction in social roles [5], and the persistence of restriction for community participation, particularly mobility outside the home, increases with age [6]. To insure a proper match between the device and the user, the importance of systematic follow-up after wheeled mobility device provision has been acknowledged by international agencies [1]. A timely follow-up allows the detection of adverse outcomes such as early stage pressure sores [7, 8], pain associated with WC use [9, 10], or WC-related incidents due to malfunction of the device [11]. However, reviews of the wheeled mobility outcome literature reveal that adverse outcomes are rarely monitored or addressed [12, 13], and available studies tend to focus on one problem at a time [14–16]. Collecting routine follow-up data encompassing multiple WC-related issues would be important and novel since healthcare teams rarely know if they are meeting their clients’ mobility needs beyond WC delivery [17–20].

Major obstacles to follow-up are the burden of assessment and limited human resources for data collection and interpretation of the results. One creative solution is the use of automated calls administered by an interactive voice response system (IVRS). An IVRS allows a computer to detect voice and/or keypad input, can ask prerecorded questions and generate audio messages through automated calls. Although the IVRS is connected to a complex electronic web-based system, the end-user accesses it with a standard telephone. Besides simplicity of use, this information technology may offer significant advantages in terms of staff workload [21] and costs [22, 23]. Thus the monitoring of WC users with an IVRS could allow rehabilitation centers to systematically follow their clients with a simple technology and ensure timely interventions only for those at higher risk during the critical months after WC prescription.

The Mobility Outcomes via Information Technologies (MOvIT) project uses an IVRS to detect older adults at risk of poor outcomes after a WC prescription and sends an electronic alert to a rehabilitation professional if needed. This paper reports on the feasibility of the MOvIT monitoring system, including the development of a questionnaire, and the early detection of adverse outcomes associated with WC use for adults above 50 years of age.

Methods

The study involved a mixed methods two-phase approach. The goal of Phase 1 was the development and face validation of the MOvIT monitoring questionnaire, while Phase 2 evaluated the feasibility of using an IVRS to administer the MOvIT questionnaire with a prospective cohort of WC users 1 and 3 months after WC delivery. The project was approved by the institutional review board of the Center for Interdisciplinary Research in Rehabilitation of Greater Montreal (CRIR-559-1110) and written informed consent was obtained from each participant.

Phase I Development and face validation of the screening questionnaire

We followed the approach suggested by Streiner and Norman [24] to develop the experimental version of the questionnaire: 1) preliminary conceptual decisions, 2) item generation and response scaling, and 3) face validation.

Preliminary conceptual decisions

The Taxonomy of Assistive Technology Device Outcomes [25] suggests generic classes of outcomes as the most pertinent for the conceptual modeling of assistive technology device intervention–outcome relationships. We used the taxonomy to operationalize our screening tool relative to the three dimensions for determining assistive technology outcome effectiveness, subjective well-being, and social significance. The outcomes were extracted from literature reviews conducted on wheeled mobility outcomes, and studies reporting on wheeled mobility adverse outcomes shown in Table 1. The specific definitions of each dimension are also provided in Table 1.

Table 1 Conceptual framework of the monitoring questionnaire
Full size table

Item generation and response scaling

The MOvIT monitoring questionnaire was structured as an algorithm with a two-step process: 1) primary questions (called filter questions) assessed if a problem was present, and 2) secondary questions (called optional questions) were administered when an adverse outcome was detected to probe if the problem was related to use of the new WC. The questions were worded to elicit a nominal response, in order to allow automated administration methods with an IVRS using voice recognition or keypad numeric entry on the phone.

For each outcome identified by the literature review, preliminary items were created or adapted from published questionnaires when available. For the effectiveness dimension, the Seating Intervention Tool (SIT) provided the preliminary pool of items to cover body functions as it was designed for the case detection of WC seating needs by nursing staff with a broad focus on aspects such as pressure, discomfort, positioning, stability, as well as mobility. Similarly, the social significance dimension items covering frequency of device use were based on the environmental continuum of the Life-Space Assessment. Items related to the other constructs were derived from the literature review as shown in Table 1 and developed specifically for the MOvIT preliminary questionnaire.

Face validation of the MOvIT questionnaire

Participants

We assessed the face validity of the preliminary questionnaire filter and optional questions with end-users (WC users and healthcare providers). The target population of WC users was composed of adults 50 years of age and older who had received a manual or power WC in the previous year. Individuals living in long-term care facilities were excluded as the IVRS requires a direct telephone line. The healthcare providers had expertise in geriatric care and/or a minimum of three years of clinical practice, including at least one year at a WC clinic.

Procedures

Semi-structured individual interviews were conducted with the end-users. To establish rapport, we collected information on background and experience with WCs, including use and adverse outcomes. We then presented them with a mock-up version of the IVRS questionnaire, using audio files to simulate an automated call based on the script of the call algorithm. We administered the same mock-up version of the questionnaire during the first wave of six interviews, and revised iterations for the second wave of six interviews. The interviewer probed with questions based on cognitive processing theory. End-users commented on their thoughts while answering the questionnaire using think-aloud techniques. To facilitate the process, we presented suggestions made by previous participants to stimulate new ideas regarding the refinement of the items. Participants were asked to propose new formulations for questions if needed.

Phase II Feasibility

Design

To assess the feasibility of the MOvIT monitoring intervention for the early detection of WC-related adverse outcomes, a prospective cohort of 65 WC users were followed at 3 time points: T0 (WC delivery), T1 (1 month post) and T2 (3 months post).

Participants

The target population was comprised of adults between the ages of 50 and 90 years who were prescribed a new manual or power WC. Individuals who were not living in the community one month after WC delivery and those who were unable to answer questions by voice or keypad were excluded.

MOvIT Monitoring questionnaire

The final MOvIT monitoring questionnaire covers nine potential adverse outcomes: 1) non-use of WC, 2) pain, 3) skin problems, 4) positioning problems, 5) WC incidents, 6) psychosocial issues, 7) restricted WC participation, 8) limited WC skills and knowledge, and 9) technical problems (Appendix 1). The questionnaire has 16 general filter questions, and 22 optional questions to probe about the severity or link with WC use. Each item comprises decision rules programmed by an IVRS provider (TelAsk) in the IVRS to determine the presence of potential WC-related adverse outcomes. The call procedure (illustrated in Fig. 1) starts with three filter questions about WC use in different environments during the preceding month. If the individual did not use the device during the month preceding the assessment, the script proceeds with a series of questions about reasons for non-use (e.g. architectural barriers or transportation issues). If the individual used the WC during the preceding month, a series of 10 filter questions about potential adverse outcomes related to WC use, 2 filter questions about psychosocial distress, and one filter question on satisfaction (5-level response scale) are administered. If no WC-related adverse outcomes are detected, respondents have the option to request to be contacted by the staff if they have other concerns. An optional menu providing WC repair procedure information through phone, email or mail is available at the end of the T2 call.

Fig. 1
figure 1

Call procedure

Full size image

Procedures

We contacted participants prior to their appointment for WC delivery at the rehabilitation center. A period of 2 weeks before or after WC delivery was tolerated to complete the baseline assessment. At baseline, the research coordinator tested an automated call to familiarize the participant with use of the keypad and voice recognition. Participants selected the preferred day of the week and time for their calls. The system was programmed to call at 1 month (T1) and 3 months (T2) post-WC delivery to administer the MOvIT questionnaire. It made 2 to 12 attempts over 4 days to reach the participant according to rules for a busy signal, no answer, answering machine, hang-up or partial call.

When a response to MOvIT items indicated there was an adverse outcome in one or more of the 9 domains assessed, a clinical coordinator (occupational therapist) from each site was notified electronically to call the subject back. This call back consultation occurred within seven working days. The coordinator accessed a web interface to see the call results and baseline information. Upon completion of the call, the coordinator confirmed each adverse outcome identified by the automated call and entered a summary note in the web-based record with clinical hypotheses about likely cause(s) and recommendations.

Semi-structured questionnaire

Individual interviews led by the first author (CA) were conducted with all participants after completion of the T2 call. The questionnaire started with a general open-ended question about the WC users’ experience with the MOvIT automated calls, followed by probes covering perceived ease of use (duration, clarity, speed, technical issues, input mode), usefulness and intention to use MOvIT monitoring in the future based on the Post-Study System Usability Questionnaire.

Feasibility

To assess feasibility we extracted data from the MOvIT database concerning call duration, call completion rate, number of cases requiring a callback, frequency and types of adverse outcomes detected by the automated call, frequency and types of adverse outcomes confirmed by the clinical coordinator, and frequency and types of recommendations.

Wheelchair user characteristics

Family income, highest level of education completed, age, sex, diagnosis, device type (manual vs power WC), and experience with WC use (first vs renewal WC) were collected at baseline.

Analyses

Phase I Development and face validation of the monitoring questionnaire

An iterative approach was used to refine the preliminary questionnaire. The analysis was split into two waves of six participants. The results from the first wave of interviews were analyzed and implemented in a revised questionnaire. During the second wave of interviews, we edited the questionnaire after each iteration until we obtained data replication or redundancy [26] regarding the questionnaire content (e.g. item formulation, additional items), as well as the best process for the monitoring intervention (e.g. timing of the calls).

Phase II Feasibility

Quantitative analyses consisted of descriptive statistics (means, SD, and/or proportions). Acceptance rate was inferred from the ratio of eligible participants who agreed to participate, and attrition rate was the proportion of enrolled participants who were lost to follow-up because they were unable to complete at least one automated call or the T2 interview. To identify items that triggered unnecessary call backs, we calculated the ratio of WC-related adverse outcomes confirmed by the clinical coordinator versus detected by the automated calls. The characteristics of participants vs. non-participants were compared using t-tests and an alpha of p < 0.05.

To develop a better understanding concerning perceived ease of use of the automated calls and usefulness, the responses to the semi-structured questionnaire were coded, regrouped into themes and occurrences were counted. Most responses were grouped along a 5-point continuum ranging from clearly unproblematic to clearly problematic. Usefulness was categorized as useful for self, others, self and others or not useful. Intention to use was dichotomized (yes or no). Summary statistics were used to present these data.

Results

Phase I Development and face validation of the screening questionnaire

Twelve end-users participated in validating the questionnaire: 4 men and 3 women WC users aged 52 to 80 years of age who had 1.5 months to 20 years of experience with manual or power WCs; 4 occupational therapists and 1 orthotist who had 9.5 to 25 years of experience with WC prescription and the geriatric population.

Table 1 shows the conceptual framework that was used to structure the development and assist with the face validation of the MOvIT monitoring questionnaire. Two types of changes were made to the preliminary questionnaire: 1) item modifications, and 2) item additions. An example of item modification consisted of modifying the conceptual definition of the environment from a continuum (using the WC in the home, neighborhood, town) to a dichotomous definition (Q5/Q6, inside/outside home) to better reflect the vocabulary commonly used by end-users. Moreover, modifications to four optional items were requested by healthcare providers to insure that cases flagged as positive would present a severe problem or a significant worsening of a pre-existing condition (Q7b, Q7d, Q15, Q17). Their concern was to avoid unnecessary call backs about pre-existing problems or other problems unrelated to use of the new WC.

The item additions were either co-constructed with end-users or selected from available questionnaires. Six new items were co-constructed with end-users, namely: use of the device during transportation (Q11), weather reasons for non-use (Q8), mechanical breakdown and deficiencies (Q20), caregiver issues (Q26), and detailed “how-to” information about WC servicing (Q21, Q39). The suggestion to add items about psychosocial issues came from WC users. Nine items (Q27-Q35) are from the Patient Health Questionnaire three-item version (PHQ-3) [27] since a simplified version for older adults was available [28] and was validated with WC users [29–31]. However, these psychosocial items required a second layer of analysis by the clinical coordinator during the telephone contact to determine if the psychosocial distress was caused by the WC or not. Finally, WC users asked to end the questionnaire on a positive note when no adverse outcomes were detected using a general satisfaction question (Q36) adapted from the Quebec User Evaluation of Satisfaction with Assistive Technology [32].

All items use a dichotomous response scale (yes or no), except for the general satisfaction question Q36, which is adapted to provide a 5-level response scale ranging from not satisfied at all to very satisfied [32] (Appendix I).

Phase II Feasibility

A total of 122 individuals were contacted; 92 were eligible, 71 were enrolled, and 65 completed the call schedule and final interview, resulting in a 77 % (71/92) acceptance rate and 92 % (65/71) completion rate. The reasons for exclusion and loss to follow-up are listed in Fig. 2 and show that the main reason for exclusion was the presence of cognitive-communication impairments. The median (SD) duration of the call was 7.0 (1.9) and 8.0 (1.7) minutes at T1 and T2 respectively.

Fig. 2
figure 2

Feasibility study flow

Full size image

The sample was composed of 36 women and 29 men with a mean age (SD) of 68.7 years (9.5). Approximately two-thirds were prescribed a manual WC (n = 42; 64.6 %) and one-third a power WC (n = 23; 35.4 %). A first WC was obtained by 63.1 % of the sample (n = 25 manual WCs; n = 16 power WCs) while 36.9 % got a renewal WC (n = 17 manual WCs; n = 7 power WCs). A majority of the participants had neurological conditions (n = 50; 76.9 %), followed by musculoskeletal (n = 13; 20 %) and medically complex conditions (n = 2; 3.1 %). Other impairments affected fine motor (n = 25; 38.5 %), speech (n = 15; 23.1 % dysarthria or aphasia) and sensory functions (n = 6; 9.2 %; n = 1 legal blindness and n = 5 auditory impairments). Participants enrolled (n = 71) did not differ significantly from those with complete data (n = 65) with regard to sex, age, device use and health conditions (t-test p > 0.16).

Table 2 reports the WC-related adverse outcomes identified by the automated calls and confirmed by the screening intervention of the clinical coordinator at T1 and T2. Out of 65 participants, 69.2 % (n = 45) required contact at one point and at least one problem was confirmed for 58.5 % (n = 38) of the sample. A total of 96 problems were identified by the automated call and 78 of them were confirmed as WC-related adverse outcomes by the clinical coordinator. The potential WC-related adverse outcomes tracked by the screening questionnaire were each identified in 2 to 26 participants during the first 3 months of monitoring. Other reasons for calling back were 15 (23.1 %) participants who reported low satisfaction and 6 (9.2 %) contacts requested by participants.

Table 2 WC-related adverse outcomes based on calls and screening by clinical coordinator (N = 65)
Full size table

The most frequently confirmed adverse outcomes were technical issues with the WC (n = 23; 35.4 %), followed by restricted WC participation, such as having difficulty doing important activities inside or outside the home (n = 13; 20.0 %), pain and discomfort (n = 13; 20.0 %), and positioning problem (n = 10; 15.4 %). Some adverse outcomes were not confirmed by the clinical coordinators because the participant pressed yes by mistake. Other questions such as psychological distress items were not confirmed due to the questionnaire design. By design, these screening questions (Q27-Q35) required a second layer of analysis by the clinical coordinator to establish causality with use of the new WC, and thus 11 out of 13 flags for psychological distress were not confirmed because the distress was unrelated to WC use (e.g. mourning for a deceased family member).

We examined the type and frequency of recommendations made by the clinical coordinator to explore the usefulness of the T1 and T2 calls. We found that 66.2 % (n = 43) of study participants received at least one recommendation during the monitoring period. Out of a total of 54 recommendations given, the most frequent were for a clinical re-evaluation (n = 16; 29.6 %) or mechanical adjustment (n = 13; 24.1 %). Some overlap was observed between the recommendations of the two calls since a total of 16 (29.6 %) recommendations were given at T1 exclusively, 23 (42.6 %) at T2 exclusively, and 15 (27.8 %) were repeated between T1 and T2. The repeated recommendations were due to the need for another appointment to resolve the problem for 53.3 % (n = 8) or to non-compliance to return to the clinic for an intervention to address the identified item of concern for 46.7 % (n = 7).

When asked to tell us about their experience with the MOvIT automated calls (Table 3), many described how they felt about being monitored (e.g. cared for, respected for their opinion) or reported what they liked (e.g. call display) or disliked (e.g. slow pace of the script) about various aspects of the system. Although designed for WC user self-report, it is interesting to note that five caregivers answered as proxy respondents. The reasons provided were: participant was unavailable, caregiver was on a shared line in case participant needed help, physical help was given to click on the keypad responses but responses were provided by WC user, and inability to understand the questions.

Table 3 Results from semi-structured usability questionnaire
Full size table

The automated call was not problematic for a majority of participants with respect to call duration (n = 48; 73.8 %), clarity (n = 55; 84.7 %), and speed (n = 51; 78.5 %). When probed about call duration, some participants mentioned that the call was a little long or too long (n = 17; 26.2 %), while others (n = 5; 7.7 %) felt that the slower pace of the dialogue caused this impression. Clarity and speed were linked since many who felt that the pace of the call was appropriate also appreciated script clarity, while a few (n = 5; 7.7 %) found that the clarity of the pronunciation was exaggerated. Some wished to speed up by skipping the directions or using the rapid pace option (n = 4; 6.2 %), while others said they needed more time to think prior to answering (n = 3; 4.6 %).

Technical issues with the automated calls were clearly unproblematic for most participants (n = 48; 73.8 %), and a majority were able to answer without any repetition (n = 43; 66.2 %). When asked, four (6.1 %) participants reported major technical problems experienced during the calls: one had a temporary disturbance with her telephone line due to reconfiguration of the intercom system in her building while three experienced voice recognition problems that led to call disconnection.

When asked about the usefulness of the monitoring intervention all participants except one had an opinion. A majority (n = 53/64; 82.8 %) felt it was useful. Participants reported it was useful for them personally (n = 32; 50.0 %), useful for both themselves and others (n = 11; 17.2 %), or useful for other WC users or the healthcare team (n = 10; 15.6 %). The reasons provided by those who thought the intervention was not useful were that no changes were required for them personally (n = 8; 12.5 %), they preferred to call their occupational therapist directly when they had a problem (n = 2; 3.1 %), or their WC-related adverse outcome was unresolved (n = 1; 1.6 %). When asked if they would register if offered this monitoring service in the future, 81.5 % (n = 53/65) of the participants reported interest in using such a service.

Discussion

The aim of this study was to develop a questionnaire for use in a phone monitoring intervention for the early detection of adverse outcomes associated with WC use and to assess its feasibility for community-living middle-aged and older adults. The questionnaire monitors nine potential adverse outcomes using automated calls and telephone consultations by occupational therapists. The results suggest it was feasible to implement the monitoring intervention as approximately 77.2 % of eligible individuals from three WC clinics participated and 92.0 % of the enrolled participants completed the protocol. A large majority of WC users (82.8 %) thought it would be a useful follow-up tool.

Lack of follow-up after prescription of a mobility device is one of the key complaints of WC users [33, 34]. There are many obstacles to the implementation of a systematic follow-up, including the poor applicability of available measures for clinical use, the geographic dispersion of users, and restricted access to post-discharge rehabilitation services [35, 36]. Feasibility is a major obstacle when attempting to integrate systematic data collections from WC users in real life clinical practices [19]. Most existing outcome measures for follow-up after WC prescription are in-depth assessments that measure single constructs such as device satisfaction [32], psychosocial impact [37], or mobility [38], and they can take up to 30 min each to administer. The MOvIT questionnaire could be used as a preliminary step in identifying those who need further follow-up or in-depth assessments. A distinctive contribution of the data collection approach we are proposing is its potential compatibility with alternative data collection methods, such as automated calls through interactive voice response systems, telephone interviews by clerical staff, or online assessments. The ultimate goals are to remove the burden from busy clinical personnel, maximize the early detection of poor WC-related outcomes to alert agencies about unmet needs, prevent the recurrence of WC-related adverse events, and relieve the healthcare system of a tremendous financial burden.

Developing the questionnaire for the systematic monitoring of community-living adults involved several challenging conceptual and methodological issues. Our first challenge was to define and select undesirable results, negative consequences or injuries resulting from WC use. Reviews of adverse outcomes are quite rare and account for only 4 % of all systematic reviews in health care [39]. Moreover, Golder at al. [39] found that these reviews typically focus on prespecified effects, which limits their scope. Thus we had to base our initial selection of WC-related adverse outcomes on published reviews that used a broad search approach. Using the Taxonomy of Assistive Technology Device Outcomes [25] to anchor these concepts proved helpful as there is no existing framework of WC-related adverse outcomes. Subsequently, the input of end-users refined our questions to include the most relevant problems that are amenable to intervention. End-users, particularly healthcare providers, insisted on detecting adverse outcomes that are clinically relevant (e.g. worsening of a pre-existing condition directly caused by the new WC) and could be ameliorable following a screening intervention (e.g. resolved by modifying the WC configuration). We encountered a challenge in trying to reconcile the suggestions made by WC users and healthcare providers, particularly around the construct of psychological distress. These questions were requested by WC users in Phase I while most healthcare providers did not feel it was part of their mandate to address psychological issues. Results of Phase II showed that only 2 out of 13 flags for general psychological distress were confirmed as WC-related. The next iterations of MOvIT will have to find the right balance to address the need for psychosocial support without adding an unnecessary burden on clinical coordinators. Finally, it is very important to focus on elements that are considered severe or high-priority and most modifiable by the target population because they are more likely to lead to behavioral change [40]. Including clinicians is recognized as an essential step to promote knowledge transfer and facilitate adoption of an innovation in clinical practice [41]. We engaged them throughout the monitoring tool development process, which may have contributed to tailoring it to their needs and to raising its overall acceptability and potential for implementation in the future.

Regarding the usefulness of the monitoring tool for early detection of WC-related adverse outcomes, we believe the 1- and 3-month calls were both necessary as some unique adverse outcomes and recommendations were identified at each time point. Our findings indicate similar proportions of WC-related adverse outcomes as reported in previous studies. For instance, the rate of pain/discomfort and positioning complaints/issues that we found (15-20 %) is comparable to the 22 % reported by Bourbonnière et al. [42]. Our estimate of a 35 % rate of technical problems is also in line with past studies reporting breakdown rates in the range of 33-45 % [16, 43, 44]. Our WC incident rate of 11 % is lower than in other publications, primarily because these studies spanned one to three years of WC use with rates of 31-38 % for falls and 14-47 % for WC injuries [15, 43, 45, 46]. The proportion of respondents who were dissatisfied with their WC in our study was also lower than for previous studies reporting after up to 18 months of WC use (23 % vs. 34 % [47]) but higher than for studies that did not control for duration of WC use (23 % vs. 11 % [48]).

Finding the right balance between respondent burden and accommodating the wide range of capacities and comorbidities present in various diagnoses in the population served by WC clinics remains challenging. As proposed by Miller et al. [49], one promising option is the concept of a self-adjusting IVRS that would automatically adapt to the respondent’s cognitive and sensory capabilities when the system has to repeat instructions. Our participants asked for options such as pause, repeat or speed adjustment. We also believe that alternative formats should be developed in the future to offer the possibility of choosing between IVRS, on-line and proxy-respondent versions with the results centralized in a single database. This would allow each individual to select the most compatible modality, as a ‘one size fits all’ approach may not be possible with a heterogeneous population.

The results of this feasibility study should be interpreted with caution because the study was designed to explore the potential utility of an IVRS for monitoring assistive technology. The enrolments took place over a short period of time between October 2011 and January 2012, meaning that most of the follow-up calls took place during the winter. Participants repeatedly mentioned winter weather as an obstacle to returning for a servicing appointment, and this should be taken into account in future studies by comparing seasonal variation in the rates of unresolved issues. A seasonal effect may explain why a fair percentage (n = 24; 36.9 %) of the sample had unresolved problems at both 1 and 3 months post-WC delivery, as some delayed their servicing visits until spring. The representativeness of the very old age group (85+ years) is also limited, possibly due to a higher prevalence of cognitive-communication impairments in this group. At the present stage, it is not possible to make inferences about the overall effectiveness of the intervention in reducing adverse events or costs following WC prescription because the study had a relatively small sample size and was not designed to address this question. However our results show that a majority of WC users accepted the technology and found it easy to use, which justifies investigating the effect of the intervention in a larger randomized controlled trial.

Conclusions

In this study we developed an intervention with end-users to monitor nine potential WC-related adverse outcomes using information technologies. It was feasible to implement the intervention in WC clinics using an IVRS connected to a web interface, and a large majority of WC users thought it would be a useful follow-up tool. Tracking adverse outcomes soon after WC prescription is a crucial aspect to give providers feedback about the needs of specific individuals, and also to identify gaps at a higher system-level. We believe that the monitoring tool will simplify that task for clinical teams.