Abstract
Objectives
To evaluate the validity and participants’ acceptance of an online assessment of role function using computer adaptive test (RF-CAT).
Methods
The RF-CAT and a set of established quality of life instruments were administered in a cross-sectional study in a panel sample (n = 444) recruited from the general population with over-selection of participants with selected self-report chronic conditions (n = 225). The efficiency, score accuracy, validity, and acceptability of the RF-CAT were evaluated and compared to existing measures.
Results
The RF-CAT with a stopping rule of six items with content balancing used 25 of the available bank items and was completed on average in 66 s. RF-CAT and the legacy tools scores were highly correlated (.64–.84) and successfully discriminated across known groups. The RF-CAT produced a more precise assessment over a wider range than the SF-36 Role Physical scale. Patients’ evaluations of the RF-CAT system were positive overall, with no differences in ratings observed between the CAT and static assessments.
Conclusions
The RF-CAT was feasible, more precise than the static SF-36 RP and equally acceptable to participants as legacy measures. In empirical tests of validity, the better performance of the CAT was not uniformly statistically significant. Further research exploring the relationship between gained precision and discriminant power of the CAT assessment is needed.
Similar content being viewed by others
References
Cella, D., Riley, W., Stone, A., et al. (2010). The patient-reported outcomes measurement information system (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology, 63(11), 1179–1194.
Gandek, B., Sinclair, S. J., Jette, A. M., & Ware Jr., J. E. (2007). Development and initial psychometric evaluation of the participation measure for post-acute care (PM-PAC). American Journal of Physical Medicine & Rehabilitation, 86(0894-9115; 1), 57–71.
Haley, S. M., Gandek, B., Siebens, H., et al. (2008). Computerized adaptive testing for follow-up after discharge from inpatient rehabilitation: II. Participation outcomes. Archives of Physical Medicine and Rehabilitation, 89(2), 275–283.
Mulcahey, M. J., Haley, S. M., Duffy, T., Pengsheng, N., & Betz, R. R. (2008). Measuring physical functioning in children with spinal impairments with computerized adaptive testing. Journal of Pediatric Orthopedics, 28(0271-6798; 3), 330–335.
Wilkie, D. J., Judge, M. K., Berry, D. L., Dell, J., Zong, S., & Gilespie, R. (2003). Usability of a computerized PAINReportIt in the general public with pain and people with cancer pain. Journal of Pain and Symptom Management, 25(0885-3924; 3), 213–224.
Bjorner, J. B., Kosinski, M., & Ware, J. E., Jr. (2003). Calibration of an item pool for assessing the burden of headaches: An application of item response theory to the headache impact test (HIT). Quality of Life Research, 12(8), 913–933.
Bayliss, M. S., Dewey, J. E., Dunlap, I., et al. (2003). A study of the feasibility of internet administration of a computerized health survey: The headache impact test (HIT). Quality of Life Research, 12(8), 953–961.
Cella, D., Yount, S., Rothrock, N., et al. (2007). The patient-reported outcomes measurement information system (PROMIS): Progress of an NIH roadmap cooperative group during its first two years. Medical Care, 45(5 Suppl 1), S3–S11.
Haley, S. M., Fragala-Pinkham, M., & Ni, P. (2006). Sensitivity of a computer adaptive assessment for measuring functional mobility changes in children enrolled in a community fitness programme. Clinical Rehabilitation, 20(7), 616–622.
Hart, D. L., Deutscher, D., Werneke, M. W., Holder, J., & Wang, Y. C. (2010). Implementing computerized adaptive tests in routine clinical practice: Experience implementing CATs. Journal of Applied Measurement, 11(3), 288–303.
Hart, D. L., Wang, Y. C., Cook, K. F., & Mioduski, J. E. (2010). A computerized adaptive test for patients with shoulder impairments produced responsive measures of function. Physical Therapy, 90(6), 928–938.
Hart, D. L., Werneke, M. W., Wang, Y. C., Stratford, P. W., & Mioduski, J. E. (2010). Computerized adaptive test for patients with lumbar spine impairments produced valid and responsive measures of function. Spine (Phila Pa 1976), 35(24), 2157–2164.
Turner-Bowker, D. M., Saris-Baglama, R. N., Anatchkova, M., & Mosen, D. M. (2010). A computerized asthma outcomes measure is feasible for disease management. The American Journal of Pharmacy Benefits, 2(2), 119–124.
Anatchkova, M. D., Saris-Baglama, R. N., Kosinski, M., & Bjorner, J. B. (2009). Development and preliminary testing of a computerized adaptive assessment of chronic pain. The Journal of Pain, 10(9), 932–943.
Becker, J., Fliege, H., Kocalevent, R. D., et al. (2008). Functioning and validity of a computerized adaptive test to measure anxiety (A-CAT). Depress Anxiety, 25(12), E182–E194.
Kopec, J. A., Badii, M., McKenna, M., Lima, V. D., Sayre, E. C., & Dvorak, M. (2008). Computerized adaptive testing in back pain: Validation of the CAT-5D-QOL. Spine (Phila Pa 1976), 33(12), 1384–1390.
Kosinski, M., Bjorner, J. B., Ware, J. E., Jr., Sullivan, E., & Straus, W. L. (2006). An evaluation of a patient-reported outcomes found computerized adaptive testing was efficient in assessing osteoarthritis impact. Journal of Clinical Epidemiology, 59(7), 715–723.
World Health Organization. (2002). Towards a common language for functioning, disability and health: ICF: The international classification of functioning, disability and health. Geneva: World Health Organization (WHO).
Anatchkova, M. D., & Bjorner, J. B. (2010). Health and role functioning: The use of focus groups in the development of an item bank. Quality of Life Research, 19(1), 111–123.
Anatchkova, M. D., Ware, J. E., & Bjorner, J. B. (2011). Assessing the factor structure of a role functioning item bank. Quality of Life Research, 20, 745–758.
Muraki, E. (1997). Generalized partial credit model. In V. D. Linden & R. K. Hambleton (Eds.), Handbook of item response theory (pp. 153–164). New York, NY: Springer.
Bjorner, J. B., Chang, C. H., Thissen, D., & Reeve, B. B. (2007). Developing tailored instruments: Item banking and computerized adaptive assessment. Quality of Life Research, 16(Suppl 1), 95–108.
Anatchkova, M. D., & Bjorner, J. B. Item calibration of a generic role functioning item bank. ISPOR 13th Annual European Congress, Prague. http://www.ispor.org/research_study_digest/details.asp.
Ware, J. E., Jr., & Dewey, J. (2000). How to score version two of the SF-36 health survey. Lincoln, RI: QualityMetric Incorporated.
Kessler, R. C., Barber, C., Beck, A., et al. (2003). The world health organization health and work performance questionnaire (HPQ). Journal of Occupational and Environmental Medicine, 45(2), 156–174.
Lerner, D., Amick III, B. C., Rogers, W. H., Malspeis, S., Bungay, K., & Cynn, D. (2001). The work limitations questionnaire. Medical Care, 39(0025-7079; 1), 72–85.
Ware, J. E., Jr., Kosinski, M., Turner-Bowker, D. M., & Gandek, B. (2002). How to score version two of the SF-12 health survey. Lincoln, RI: QualityMetric Incorporated.
Moriarty, D. G., Zack, M. M., & Kobau, R. (2003). The centers for disease control and prevention’s healthy days measures: Population tracking of perceived physical and mental health over time. Health and Quality of Life Outcomes, 1(1), 37.
McHorney, C. A., Ware, J. E., Jr., & Raczek, A. E. (1993). The MOS 36-item short-form health survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Medical Care, 31, 247–263.
Acknowledgments
The project described was supported by Award Number K01AG028760 from the National Institute on Aging. Partial salary support for Dr. Anatchkova is provided by the National Institutes of Health grant 1U01HL105268-01.The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Aging or the National Institutes of Health.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Anatchkova, M., Rose, M., Ware, J. et al. Evaluation of a role functioning computer adaptive test (RF-CAT). Qual Life Res 22, 1085–1092 (2013). https://doi.org/10.1007/s11136-012-0215-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11136-012-0215-6