Abstract
A recently developed computerized Animated Activity Questionnaire (AAQ) uses video animations to measure limitations in basic daily activities in patients with hip or knee osteoarthritis (OA). The AAQ is less time and money consuming than performance-based tests and less burdensome for patients. Compared to patient-reported outcome measurements, the AAQ is less influenced by the perception of the patient. Previous validation studies of the AAQ showed promising results. Since the AAQ purports to measure activity limitations in real-life situations, this study aims to establish the construct validity of the AAQ by comparing the AAQ to home-recorded videos. Home-recorded videos of eleven basic activities performed by 22 OA patients were transformed into scores and subsequently correlated with the AAQ, the Hip disability and Knee injury Osteoarthritis Outcome Score (H/KOOS), and three performance-based tests (Stair Climbing Test, Timed Up and Go test, and the 30 s Chair Stand Test). The AAQ was expected to correlate highly with the home-recorded videos and the performance-based tests, and moderately with the H/KOOS. As hypothesized, the AAQ correlated highly with the home-recorded videos and performance-based tests (0.83 and, 0.73 respectively), but also unexpectedly highly with the H/KOOS (0.79). In conclusion, since the AAQ correlates highly with home-recorded videos, it seems to have potential as a measurement tool to assess limitations in activities close to the real-life situation, but this should be confirmed in a larger sample of patients.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Osteoarthritis (OA) is the most prevalent chronic joint disease worldwide, and one of the leading causes of disability and activity limitations in daily life [1, 2]. Hip and knee are two of the most affected joints [2]. An essential key in the management of hip and knee OA is a comprehensive assessment of physical limitations in daily activities [3]. Ideally, we would like to measure the level of performance of activities in daily life such as a person’s home situation. But for several logistic and privacy-related reasons, this is not possible. In general, there are two types of alternative measurements: patient-reported outcome measures (PROMs) and performance-based tests, which are both recommended to assess the patients’ level of activity limitations [4, 5]. PROMs rely on self-report and measure the experienced degree of difficulty in performing daily activities. They are considered easy to implement, inexpensive, and harmless for the patient. Nevertheless, PROMs are highly subjective to the perception of the term difficulty and to the adopted reference frame of the patient. PROM scores are influenced by a large number of personal factors (e.g., BMI, depression, self-efficacy, and pain) [6–8]. Furthermore, Fayers et al. [9] signify that some patients relate to the time before they got sick while others compare themselves to healthy peers. Furthermore, the activities referred to in the questionnaire are subject to the reference frame of the respondent [10]. For example, the perception of stair climbing highly depends on what kind of stairs a subject has in mind.
Performance-based tests, on the other hand, are associated with different limitations. They are considered expensive, time–consuming, and invasive as they demand trained personnel, test facilities, and physical presence of the patient [11]. It is also advocated that tests administered in the clinic, essentially, do not represent a real-life situation and only capture a snapshot of reality [12]. In addition to this, performance-based scores are more sensitive to observer bias, instruction bias, and the motivation of the participant to perform the test.
Research in the field encourages a complementary use of both outcome measures [6, 13, 14]. In fact, it is suggested that PROMs and performance-based test measure different constructs. Performance-based tests evaluate the maximum capacity of a patient in performing isolated tasks, while PROMs indicate the experienced degree of difficulty (i.e., a perception) [15].
No measurement tool exists that measures the actual level of performance of an activity. In light of this, the computer-administered Animated Activity Questionnaire (AAQ) was developed [16, 17]. The AAQ contains seventeen items with every item representing one basic daily activity. With the AAQ, patients watch animated videos on a computer, in which a dummy (an animation of a person) performs different activities, such as stair climbing and rising from a chair, in several ways (each way exposing a different level of difficulty). An example can be seen on: http://kmin-vumc.nl/_14_0.html. Patients are asked to select the animation that best represents their way of performing the activity.
We expect that by showing videos of performance levels instead of asking about the degree of experienced difficulty, the subjective interpretation of questions is minimized and the AAQ measures more closely the patient’s actual performance of activities in daily life.
Results of two validation studies which tested preliminary versions of the AAQ against PROMs and performance-based tests were promising [16, 17]. Currently, the cross-cultural validity of the AAQ is studied in a large international project including over 1,000 patients in seven European countries. However, since the AAQ purports to measure the performance of activities in daily life, comparisons with existing PROMs and performance-based tests will not be sufficient enough to support construct validity.
The aim of this study was to assess construct validity of the AAQ by comparing the AAQ to home-recorded videos of daily activities. Based on the construct of the AAQ, a strong correlation between the AAQ and home-recorded videos was hypothesized. Based on the fact that the AAQ measures activity limitations and not the perception of how the activities are performed, it is hypothesized that the AAQ is moderately correlated with the H/KOOS.
Method
Subjects
Patients were selected from two different centers: a rehabilitation centre in Amsterdam (Reade) and a physiotherapy practice in Arnhem (Formupgrade). Hip or knee OA patients were approached if they were previously (up to 12 months ago) or currently under treatment or had participated in earlier research projects (up to 12 months ago) at Reade. They were approached by phone and provided with some general information regarding the project. If a patient was interested, an information leaflet and an informed consent form were sent.
To be eligible, patients had to meet the following criteria: a diagnosis of hip or knee OA according to the clinical ACR criteria [18, 19], established by a medical doctor, general practitioner, or physiotherapist; a fluent understanding of the Dutch language and access to stairs at home or in the proximate area. In order to create a heterogeneous sample of OA patients, also patients who underwent joint replacement surgery were included in the study.
Procedure
After informed consent was obtained, home appointments were scheduled for every individual participant. First, patients completed the AAQ and the H/KOOS ADL subscale. Both were administered at the computer in a consecutive order. Since the questionnaires are self-explanatory, no additional instructions were provided. Subsequently, videos were taken of eleven basic daily activities: rising from a chair; sitting down on a chair; walking six meters after 15 min sitting down; ascending stairs; descending stairs; sitting down on a couch; rising from a couch; rising from a toilet; sitting down on a toilet; picking something (i.e., a small coffee mug) up from the floor, and rising from the floor. Patients were instructed to perform these activities in their own home in a consecutive order and to behave as natural as possible. For this purpose, 11 of the 17 AAQ activities were selected. For practical reasons, taking of shoes and putting on shoes were not selected. After all videos were recorded, the respondents were asked to execute three performance-based tests.
Ethical approval
This study was approved by the Medical Ethics Committee of the VU University Medical Center, Amsterdam, and conducted in accordance with the Handbook for Good Clinical Research Practice of the World Health Organization and Declaration of Helsinki principles [http://www.wma.net/en/30publications/10policies/b3/]. All patients provided written informed consent.
Measurements
The AAQ contains seventeen activities with between three and five videos per activity. The first video of each activity represents the most optimal performance. With increasing video number, the level of difficulty in performance rises. Figures 1 and 2 show two examples, one for males and one for females. Patients are asked to choose the video that best matches their own performance of the activity. All items also offer the response option ‘unable to perform.’ Scores of the AAQ were transformed into scores on a 0–100 scale with high scores corresponding to high levels of functioning. On average, the AAQ takes 10–15 min to complete.
The home-recorded videos were independently analyzed by two physiotherapists experienced in the treatment of hip and knee OA patients. Both were not involved in the data collection and were blinded to the results of the AAQ scores, H/KOOS ADL subscale scores, and performance-based tests scores. A five-point rating scale was used to score the home-recorded videos, and the score was subsequently transformed to a 0–100 scale, with a higher score corresponding to a higher level of functioning. The five-point scale was based on the International Classification of Functioning, disability, and health (ICF) for physical impairment [15]. However, some small adjustments were made for this specific purpose. The following rating system was used: a score of (1) point indicated no problems; (2) points indicated mild problems; (3) points indicated moderate problems; (4) points indicated severe problems; and (5) points were scored if a subject was not able to perform the activity.
Details with regard to the development of the AAQ are published elsewhere [16]. A description of the activities and levels of difficulty is provided in the appendix (‘Appendix’).
Directly after completing the AAQ, patients were asked to complete a PROM; the Function, Daily Living (ADL) subscale of the HOOS [20] (i.e., Hip disability Osteoarthritis Outcome Score) or KOOS [21] (i.e., Knee injury Osteoarthritis Outcome Score). The H/KOOS ADL subscale contains seventeen physical activity-related items. Each item was rated on a five-point rating scale (i.e., 0–4). Scores were transformed in 0–100 score with higher scores corresponding to higher levels of functioning. Patients were instructed to reflect on their average disability during the last week. The applied computer program did not allow for any missing values to occur. Subsequently, patients were requested to rate their hip- or knee-related pain (i.e., on average during the last week) on a numeric rating scale, with 0 corresponding to no pain and 10 to the worst imaginable pain.
Finally, patients were asked to execute three performance-based tests in the following predetermined order: the stair climbing test (SCT) [11, 22–24], the timed up and gotest (TUG) [22, 23, 25, 26], and the 30-s repeated chair stand test (CST) [22, 23, 27, 28]. The SCT entails the timing of ascending and descending a flight of nine stairs. The TUG comprises the timing of standing up from a chair, walk three meters, turn around, walk back, and sit down again. During the CST, subjects were requested to rise and sit down on a chair as often as possible (without the support of their hands), and the number of stands within 30 s was scored.
Statistical analysis
Descriptive statistics with regard to gender, age, joint affected (i.e., knee, hip, or both), total joint surgery (none, unilateral, bilateral), and physiotherapy treatment were presented in order to describe the study population.
Correlations were calculated between the AAQ and the home-recorded videos, the H/KOOS ADL subscale, and performance-based tests. In order to test the validity of the AAQ, the following hypothesis was formulated a priori:
-
1.
The total score of the AAQ correlates highly (>0.60) with the total score of the home-recorded videos.
In order to compare correlations between the AAQ and home-recorded videos with the correlation between the AAQ and existing measurement instruments, the following additional hypotheses for validity were formulated:
-
2.
The total score of the AAQ correlates highly (>0.60) with the total score of the three performance-based tests.
-
3.
The total score of the AAQ correlates moderately (0.3–0.6) with the total score of the H/KOOS ADL subscale.
-
4.
The correlation between the total score of the AAQ and the total score of the home-recorded videos is at least 0.10 point higher as compared to the correlation between the total score of the AAQ and the total score of the performance-based tests.
-
5.
The correlation between the total score of the AAQ and the total score of the home-recorded videos is at least 0.20 point higher as compared to the correlation between the total score of the AAQ and total score of the H/KOOS ADL subscale.
Correlations between the AAQ and the home-recorded videos, H/KOOS ADL subscale, and performance-based tests were calculated by means of the Spearman correlation coefficients. For hypotheses 2 and 4, the correlation was calculated between the AAQ and the average score of the three performance-based tests. In order to adequately average these correlations, scores of the three different performance-based tests (i.e., SCT, TUG, and CST) were transformed into Fisher’s Z scores. After summarizing and averaging the three scores, the score was transferred back into a correlation coefficient.
Results
Twenty-two patients participated in this study. Characteristics of the study population are presented in Table 1.
For one subject, the video quality of the activity sitting down on a chair was not sufficient. The score for this activity was considered a missing value. The researchers assume that this missing occurred completely at random. The missing value was replaced by the mean value of the scores of the other twenty-one participants.
In Table 2, the correlations between the AAQ scores, the home-recorded video scores, the H/KOOS ADL subscale scores, and the scores of the performance-based tests are presented.
The results in relation to our hypotheses were as follows: (1) The AAQ showed a high correlation of 0.83 with the home-recorded videos, which confirmed the hypothesis. (2) The total score of the AAQ correlated highly (0.73) with the average score of the performance-based tests, as expected (>0.60). (3) The correlation between the total score of the AAQ and the total score of the H/KOOS ADL subscale was higher (0.79) than the expected moderate correlation (0.3–0.6). (4) The difference in correlation between the AAQ and the home-recorded videos (0.83), on the one hand, and the AAQ and the performance-based tests (0.73), on the other hand, was 0.10, just as expected (difference at least 0.10 point). (5) The difference in correlation between the AAQ and the home-recorded videos (0.83), on the one hand, and AAQ and the H/KOOS ADL subscale (0.79), on the other hand, was 0.03, not as large as expected (difference at least 0.20 point).
Discussion
The AAQ is a computerized animated measurement tool that uses videos to assess activity limitations in patients with hip and knee OA. An earlier study focused on a comparison of the AAQ with performance-based tests and self-reported questionnaires [17]. This study aimed to investigate the construct validity of the AAQ by comparing it with the patients’ real-life situation (i.e., home-recorded videos). As hypothesized, the AAQ correlated highly (0.83) with the home-recorded videos, which supports the idea that the AAQ provides an adequate representation of the performance of basic daily activities in real life. The correlation of the AAQ with the performance-based tests was also high (0.73), but the high correlation (0.79) of the AAQ with the H/KOOS ADL subscale was unexpected.
Yet, some methodological limitations might provide an explanation for the high correlations of the AAQ with the H/KOOS ADL subscale.
All tests were administered at home within a short time frame, which might elucidate small differences in correlations between the AAQ scores and scores on the home-recorded videos, the performance-based tests, and H/KOOS, respectively. The high correlation between the AAQ and the H/KOOS was also found in our earlier study [16]. In both studies, patients completed the H/KOOS directly after completing the AAQ. Patients might have been influenced by the AAQ while completing the H/KOOS ADL subscale, through the visual imprinting effect of the animations. This limitation can be tackled by changing the order of instrument administration, preferably by randomization, in future validation studies. Furthermore, the activities and number of difficulty levels of the AAQ show a strong resemblance with those of the H/KOOS ADL subscale. Another limitation of this study is the small group of 22 patients. Due to logistics and small financial resources, the minimum of 30 patients according to the COSMIN standards [29] could not be met. Therefore, conclusions must be interpreted with caution and the study must be repeated in a larger sample size. Finally, no inter-rater reliability was tested in the observers while judging scores on the home-recorded videos.
The application of video animations for the measurement of activity limitations remains an unexplored area. Recently, Rejeski and colleagues developed a measurement instrument that uses video animations to describe a large set of physical challenges [30]. They also reported high correlations (0.58 and 0.59) between the video animation and performance-based tests, which supports the idea that video animations can be a good alternative for performance-based tests. Compared to the AAQ, however, this instrument contains an essential difference. For every activity, only one video is shown. Subsequently, the patient is asked whether or how many times he or she is able to repeat the displayed performance. Rejeski et al. also provided evidence that varying contextual features (i.e., stair climbing with or without a handrail) influences persons’ perception of their abilities [10]. We expect that by showing similar contextual features in different levels of performance of each activity, the AAQ will be less influenced by perception.
Besides the unique construct of the AAQ, there are other features that support a clinical implementation. In comparison with performance-based tests, the AAQ is less exposed to observer bias, bias by different test instructors, and the motivation of the patient to perform the test. And since all items refer to the average performance during the last week, the AAQ does not cover only one moment in time, as performance-based tests. Furthermore, the AAQ is considered relatively cheap, easy to implement, and less burdensome for the patient.
When compared to PROMs, the AAQ also has additional benefits. The AAQ refers to videos of the performed activities instead of the word difficulty which reduces the impact of different reference frames along with ambiguities regarding the interpretation of the term difficulty. Moreover, all videos offer a standardized virtual reality environment, which reduces the impact of different environments on the subjects’ rating. Additionally, only a limited amount of text is included, which facilitates an international implementation of the AAQ, since minimal translation is needed. Finally, minimal reading ability is required.
Nevertheless, as the AAQ relies on self-report, it will be influenced by some personal factors such as self-confidence, self-efficacy, and the ability to self-reflect. The standardized virtual environment displayed in the video is not always specific to the subjects’ home situation. And it is debatable if all patients are able to self-reflect in completing a questionnaire based on video animations.
Before the AAQ can be implemented in clinical practice, more research is necessary. First, the structural validity has to be established by use of factor analysis or item response theory analyses. Next, a larger study with a more optimal design should be carried out in order to investigate the role of potential contamination between results of the AAQ and H/KOOS ADL subscale. For instance, the order in which the AAQ and H/KOOS was administered could be reversed or randomized in a future study. Furthermore, the performance-based tests need to be administered in the clinic and not in a subjects’ home environment. Since the AAQ has a potential value for international use because minimal translation is needed, further research should focus on cross-cultural validity.
In conclusion, since the AAQ correlates highly with home-recorded videos, it seems to have potential as a measurement tool to assess limitations in activities close to the real-life situation, although this should be confirmed in a larger study.
References
Bijlsma JW, Berenbaum F, Lafeber FP (2011) Osteoarthritis: an update with relevance for clinical practice. Lancet 18:2115–2126
Pereira D, Peleteiro B, Araújo J, Branco J, Santos RA, Ramos E (2011) The effect of osteoarthritis definition on prevalence and incidence estimates: a systematic review. Osteoarthr Cartil 19:1270–1285
Dobson F, Hinman RS, Hall M, Terwee CB, Roos EM, Bennell KL (2012) Measurement properties of performance-based measures to assess physical function in hip and knee osteoarthritis: a systematic review. Osteoarthr Cartil 20:1548–1562
Gandhi R, Tsvetkov D, Davey JR, Syed KA, Mahomed NN (2009) Relationship between self-reported and performance-based tests in a hip and knee joint replacement population. Clin Rheumatol 28:253–257
Stratford PW, Kennedy DM (2006) Performance measures were necessary to obtain a complete picture of osteoarthritic patients. J Clin Epidemiol 59:160–167
Maly MR, Cosigan PA, Olney SJ (2006) Determinants of self-report outcome measures in people with knee osteoarthritis. Arch Phys Med Rehabil 87:96–104
Terwee CB, van der Slikke RMA, van Lummel RC, Bennink JB, Meijers WGH, de Vet HCW (2006) Self-reported Physical Functioning was more Influenced by Pain than Performance-Based Physical Functioning in Knee-Osteoarthritis Patients. J Clin Epidemiol 59:724–731
Watson D, Pennebaker JW (1989) Health complaints, stress, and distress: exploring the central role of negative affectivity. Psychol Rev 96:234–254
Fayers PM, Langston AL, Robertson C (2007) Implicit self-comparisons against others could bias quality of life assessments. J Clin Epidemiol 60:1034–1039
Marsh AP, Ip EH, Barnard RT, Wong YL, Rejeski WJ (2011) Using video animation to assess mobility in older adults. J Gerontol A Biol Sci Med Sci 66:217–227
Kennedy D, Stratford PW, Pagura SM, Walsh M, Woodhouse LJ (2002) Comparison of gender and group differences in self-report and physical performance measures in total hip and knee arthroplasty candidates. J Arthoplasty 17:70–77
Wittink H, Rogers W, Sukiennik A, Carr DB (2003) Physical functioning: self-report and performance measures are related but distinct. Spine 28:2407–2413
Coman L, Richardson J (2006) Relationship between self-report and performance measures of function: a systematic review. Can J Aging 25:253–270
Stevens-Lapsley JE, Schenkman ML, Dayton MR (2011) Comparison of self-reported knee injury and osteoarthritis outcome score to performance measures in patients after total knee arthroplasty. PMR 3:541–549
World Health Organization (2001) Towards a common language for functioning. Disability and Health ICF, Geneva
Peter WF, Loos M, de Vet HCW et al (2014) Development and preliminary testing of a computerized Animated Activity Questionnaire (AAQ) in patients with hip and knee osteoarthritis. Arthritis Care Res (Hoboken). doi:10.1002/acr.22386
Terwee CB, Coopmans C, Peter WF et al (2014) Development and validation of a computer-administered Animated Activity Questionnaire (AAQ) to measure physical functioning of patients with hip or knee osteoarthritis. Phys Ther 94:251–261
Altman R, Alarcon G, Appelrouth D et al (1991) The American College of Rheumatology criteria for the classification and reporting of osteoarthritis of the hip. Arthritis Rheum 34:505–514
Altman R, Asch E, Bloch D et al (1986) Development of criteria for the classification and reporting of osteoarthritis: classification of osteoarthritis of the knee. Diagnostic and Therapeutic Criteria Committee of the American Rheumatism Association. Arthritis Rheum 29:1039–1049
de Groot IB, Reijman M, Terwee CB et al (2007) Validation of the Dutch version of the Hip disability and Osteoarthritis Outcome Score. Osteoarthr Cartil 15:109
de Groot IB, Favejee MM, Reijman M, Verhaar JA, Terwee CB (2008) The Dutch version of the Knee Injury and Osteoarthritis Outcome Score: a validation study. Health Qual Life Outcomes 6:16
Bennel K, Dobson F, Hinman R (2011) Measures of Physical Performance Assessments; Self-Paced Walk Test (SPWT), Stair Climb Test (SCT), Six-Minute Walk Test (6MWT), Chair Stand Test (CST), Timed Up & Go (TUG), Sock Test, Lift and Carry Test (LCT), and Car Task. Arthritis Care Res 11:350–370
Dobson F, Hinman RS, Roos EM et al (2013) OARSI recommended performance-based tests to assess physical function in people diagnosed with hip or knee osteoarthritis. Osteoarthr Cartil 21:1042–1052
Rejeski WJ, Ettinger WH Jr, Schumaker S, James P, Burns R, Elam JT (1995) Assessing performance-related disability in patients with knee osteoarthritis. Osteoarthr Cartil 3:157–167
Steffen TM, Hacker TA, Mollinger L (2002) Age-and gender-related test performance in community-dwelling elderly people: six-Minute Walk Test, Berg Balance Scale, Timed Up & Go Test, and gait speeds. Phys Ther 82:128–137
Stratford PW, Kennedy DM, Woodhouse LJ (2006) Performance measures provide assessments of pain and function in people with advanced osteoarthritis of the hip or knee. Phys Ther 86:1489–1496
Gill S, McBurney H (2008) Reliability of performance-based measures in people awaiting joint replacement surgery of the hip or knee. Physiother Res Int 13:141–152
Jones CJ, Rikli RE, Beam WC (1999) A 30-s chair-stand test as a measure of lower body strength in community-residing older adults. Res Q Exerc Sport 70:113
Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, Bouter LM, de Vet HC (2010) The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol 10:22
Rejeski WJ, Ip EH, Marsh AP, Barnard RT (2010) Development and validation of a video-animated tool for assessing mobility. J Gerontol A Biol Sci Med Sci 65:664–671
Acknowledgments
This work was supported by Anna Foundation|NOREF.
Conflict of interest
All the authors were involved in the development of the AAQ.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
See Table 3.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.
About this article
Cite this article
Peter, W.F., Loos, M., van den Hoek, J. et al. Validation of the Animated Activity Questionnaire (AAQ) for patients with hip and knee osteoarthritis: comparison to home-recorded videos. Rheumatol Int 35, 1399–1408 (2015). https://doi.org/10.1007/s00296-015-3230-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00296-015-3230-4