Skip to main content
SpringerLink
Log in

Evaluation of a role functioning computer adaptive test (RF-CAT)

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Objectives

To evaluate the validity and participants’ acceptance of an online assessment of role function using computer adaptive test (RF-CAT).

Methods

The RF-CAT and a set of established quality of life instruments were administered in a cross-sectional study in a panel sample (n = 444) recruited from the general population with over-selection of participants with selected self-report chronic conditions (n = 225). The efficiency, score accuracy, validity, and acceptability of the RF-CAT were evaluated and compared to existing measures.

Results

The RF-CAT with a stopping rule of six items with content balancing used 25 of the available bank items and was completed on average in 66 s. RF-CAT and the legacy tools scores were highly correlated (.64–.84) and successfully discriminated across known groups. The RF-CAT produced a more precise assessment over a wider range than the SF-36 Role Physical scale. Patients’ evaluations of the RF-CAT system were positive overall, with no differences in ratings observed between the CAT and static assessments.

Conclusions

The RF-CAT was feasible, more precise than the static SF-36 RP and equally acceptable to participants as legacy measures. In empirical tests of validity, the better performance of the CAT was not uniformly statistically significant. Further research exploring the relationship between gained precision and discriminant power of the CAT assessment is needed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Cella, D., Riley, W., Stone, A., et al. (2010). The patient-reported outcomes measurement information system (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology, 63(11), 1179–1194.

    Article  PubMed  Google Scholar 

  2. Gandek, B., Sinclair, S. J., Jette, A. M., & Ware Jr., J. E. (2007). Development and initial psychometric evaluation of the participation measure for post-acute care (PM-PAC). American Journal of Physical Medicine & Rehabilitation, 86(0894-9115; 1), 57–71.

    Google Scholar 

  3. Haley, S. M., Gandek, B., Siebens, H., et al. (2008). Computerized adaptive testing for follow-up after discharge from inpatient rehabilitation: II. Participation outcomes. Archives of Physical Medicine and Rehabilitation, 89(2), 275–283.

    Article  PubMed  Google Scholar 

  4. Mulcahey, M. J., Haley, S. M., Duffy, T., Pengsheng, N., & Betz, R. R. (2008). Measuring physical functioning in children with spinal impairments with computerized adaptive testing. Journal of Pediatric Orthopedics, 28(0271-6798; 3), 330–335.

    Google Scholar 

  5. Wilkie, D. J., Judge, M. K., Berry, D. L., Dell, J., Zong, S., & Gilespie, R. (2003). Usability of a computerized PAINReportIt in the general public with pain and people with cancer pain. Journal of Pain and Symptom Management, 25(0885-3924; 3), 213–224.

    Google Scholar 

  6. Bjorner, J. B., Kosinski, M., & Ware, J. E., Jr. (2003). Calibration of an item pool for assessing the burden of headaches: An application of item response theory to the headache impact test (HIT). Quality of Life Research, 12(8), 913–933.

    Article  PubMed  Google Scholar 

  7. Bayliss, M. S., Dewey, J. E., Dunlap, I., et al. (2003). A study of the feasibility of internet administration of a computerized health survey: The headache impact test (HIT). Quality of Life Research, 12(8), 953–961.

    Article  PubMed  CAS  Google Scholar 

  8. Cella, D., Yount, S., Rothrock, N., et al. (2007). The patient-reported outcomes measurement information system (PROMIS): Progress of an NIH roadmap cooperative group during its first two years. Medical Care, 45(5 Suppl 1), S3–S11.

    Article  PubMed  Google Scholar 

  9. Haley, S. M., Fragala-Pinkham, M., & Ni, P. (2006). Sensitivity of a computer adaptive assessment for measuring functional mobility changes in children enrolled in a community fitness programme. Clinical Rehabilitation, 20(7), 616–622.

    Article  PubMed  Google Scholar 

  10. Hart, D. L., Deutscher, D., Werneke, M. W., Holder, J., & Wang, Y. C. (2010). Implementing computerized adaptive tests in routine clinical practice: Experience implementing CATs. Journal of Applied Measurement, 11(3), 288–303.

    PubMed  Google Scholar 

  11. Hart, D. L., Wang, Y. C., Cook, K. F., & Mioduski, J. E. (2010). A computerized adaptive test for patients with shoulder impairments produced responsive measures of function. Physical Therapy, 90(6), 928–938.

    Article  PubMed  Google Scholar 

  12. Hart, D. L., Werneke, M. W., Wang, Y. C., Stratford, P. W., & Mioduski, J. E. (2010). Computerized adaptive test for patients with lumbar spine impairments produced valid and responsive measures of function. Spine (Phila Pa 1976), 35(24), 2157–2164.

    Google Scholar 

  13. Turner-Bowker, D. M., Saris-Baglama, R. N., Anatchkova, M., & Mosen, D. M. (2010). A computerized asthma outcomes measure is feasible for disease management. The American Journal of Pharmacy Benefits, 2(2), 119–124.

    PubMed  Google Scholar 

  14. Anatchkova, M. D., Saris-Baglama, R. N., Kosinski, M., & Bjorner, J. B. (2009). Development and preliminary testing of a computerized adaptive assessment of chronic pain. The Journal of Pain, 10(9), 932–943.

    Article  PubMed  Google Scholar 

  15. Becker, J., Fliege, H., Kocalevent, R. D., et al. (2008). Functioning and validity of a computerized adaptive test to measure anxiety (A-CAT). Depress Anxiety, 25(12), E182–E194.

    Article  PubMed  Google Scholar 

  16. Kopec, J. A., Badii, M., McKenna, M., Lima, V. D., Sayre, E. C., & Dvorak, M. (2008). Computerized adaptive testing in back pain: Validation of the CAT-5D-QOL. Spine (Phila Pa 1976), 33(12), 1384–1390.

    Google Scholar 

  17. Kosinski, M., Bjorner, J. B., Ware, J. E., Jr., Sullivan, E., & Straus, W. L. (2006). An evaluation of a patient-reported outcomes found computerized adaptive testing was efficient in assessing osteoarthritis impact. Journal of Clinical Epidemiology, 59(7), 715–723.

    Article  PubMed  Google Scholar 

  18. World Health Organization. (2002). Towards a common language for functioning, disability and health: ICF: The international classification of functioning, disability and health. Geneva: World Health Organization (WHO).

    Google Scholar 

  19. Anatchkova, M. D., & Bjorner, J. B. (2010). Health and role functioning: The use of focus groups in the development of an item bank. Quality of Life Research, 19(1), 111–123.

    Article  PubMed  Google Scholar 

  20. Anatchkova, M. D., Ware, J. E., & Bjorner, J. B. (2011). Assessing the factor structure of a role functioning item bank. Quality of Life Research, 20, 745–758.

    Article  PubMed  Google Scholar 

  21. Muraki, E. (1997). Generalized partial credit model. In V. D. Linden & R. K. Hambleton (Eds.), Handbook of item response theory (pp. 153–164). New York, NY: Springer.

    Chapter  Google Scholar 

  22. Bjorner, J. B., Chang, C. H., Thissen, D., & Reeve, B. B. (2007). Developing tailored instruments: Item banking and computerized adaptive assessment. Quality of Life Research, 16(Suppl 1), 95–108.

    Article  PubMed  Google Scholar 

  23. Anatchkova, M. D., & Bjorner, J. B. Item calibration of a generic role functioning item bank. ISPOR 13th Annual European Congress, Prague. http://www.ispor.org/research_study_digest/details.asp.

  24. Ware, J. E., Jr., & Dewey, J. (2000). How to score version two of the SF-36 health survey. Lincoln, RI: QualityMetric Incorporated.

    Google Scholar 

  25. Kessler, R. C., Barber, C., Beck, A., et al. (2003). The world health organization health and work performance questionnaire (HPQ). Journal of Occupational and Environmental Medicine, 45(2), 156–174.

    Article  PubMed  Google Scholar 

  26. Lerner, D., Amick III, B. C., Rogers, W. H., Malspeis, S., Bungay, K., & Cynn, D. (2001). The work limitations questionnaire. Medical Care, 39(0025-7079; 1), 72–85.

  27. Ware, J. E., Jr., Kosinski, M., Turner-Bowker, D. M., & Gandek, B. (2002). How to score version two of the SF-12 health survey. Lincoln, RI: QualityMetric Incorporated.

    Google Scholar 

  28. Moriarty, D. G., Zack, M. M., & Kobau, R. (2003). The centers for disease control and prevention’s healthy days measures: Population tracking of perceived physical and mental health over time. Health and Quality of Life Outcomes, 1(1), 37.

    Article  PubMed  Google Scholar 

  29. McHorney, C. A., Ware, J. E., Jr., & Raczek, A. E. (1993). The MOS 36-item short-form health survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Medical Care, 31, 247–263.

    Article  PubMed  CAS  Google Scholar 

Download references

Acknowledgments

The project described was supported by Award Number K01AG028760 from the National Institute on Aging. Partial salary support for Dr. Anatchkova is provided by the National Institutes of Health grant 1U01HL105268-01.The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Aging or the National Institutes of Health.

Author information

Authors and Affiliations

  1. Department of Quantitative Health Sciences, University of Massachusetts Medical School, 55 Lake Avenue North, Worcester, MA, 01665-002, USA

    M. Anatchkova, M. Rose & J. Ware

  2. Medical School, Charité, University Medicine Berlin, Berlin, Germany

    M. Rose

  3. National Institute of Occupational Health, Copenhagen, Denmark

    J. B. Bjorner

  4. i3 QualityMetric Inc, Lincoln, RI, USA

    J. B. Bjorner

  5. John Ware Research Group Inc, Worcester, MA, USA

    J. Ware

Authors
  1. M. Anatchkova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Rose
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. Ware
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. B. Bjorner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Anatchkova.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Anatchkova, M., Rose, M., Ware, J. et al. Evaluation of a role functioning computer adaptive test (RF-CAT). Qual Life Res 22, 1085–1092 (2013). https://doi.org/10.1007/s11136-012-0215-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-012-0215-6

Keywords

Search

Navigation