Skip to main content

Development of an item bank and computer adaptive test for role functioning

Quality of Life Research volume 21, pages 1625–1637 (2012)Cite this article

Abstract

Objectives

Role functioning (RF) is a key component of health and well-being and an important outcome in health research. The aim of this study was to develop an item bank to measure impact of health on role functioning.

Methods

A set of different instruments including 75 newly developed items asking about the impact of health on role functioning was completed by 2,500 participants. Established item response theory methods were used to develop an item bank based on the generalized partial credit model. Comparison of group mean bank scores of participants with different self-reported general health status and chronic conditions was used to test the external validity of the bank.

Results

After excluding items that did not meet established requirements, the final item bank consisted of a total of 64 items covering three areas of role functioning (family, social, and occupational). Slopes in the bank ranged between .93 and 4.37; the mean threshold range was −1.09 to −2.25. Item bank-based scores were significantly different for participants with and without chronic conditions and with different levels of self-reported general health.

Conclusions

An item bank assessing health impact on RF across three content areas has been successfully developed. The bank can be used for development of short forms or computerized adaptive tests to be applied in the assessment of role functioning as one of the common denominators across applications of generic health assessment.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

39,95 €

Price includes VAT (Netherlands)

Fig. 1

References

  1. World Health Organization (2002). Towards a common language for functioning, disability and health: ICF—The international classification of functioning, disability and health. Geneva: World Health Organization (WHO).

  2. Merikangas, K. R., Ames, M., Cui, L., et al. (2007). The impact of comorbidity of mental and physical conditions on role disability in the US adult household population. Archives of General Psychiatry, 64(10), 1180–1188.

    Article  PubMed  Google Scholar 

  3. Bjorner, J. B., Chang, C. H., Thissen, D., & Reeve, B. B. (2007). Developing tailored instruments: Item banking and computerized adaptive assessment. Quality of Life Research, 16(Suppl 1), 95–108.

    Article  PubMed  Google Scholar 

  4. Rose, M., Bjorner, J. B., Becker, J., Fries, J. F., & Ware, J. E. (2008). Evaluation of a preliminary physical function item bank supported the expected advantages of the patient-reported outcomes measurement information system (PROMIS). Journal of Clinical Epidemiology, 61(1), 17–33.

    Article  PubMed  CAS  Google Scholar 

  5. Anatchkova, M. D., & Bjorner, J. B. (2010). Health and role functioning: The use of focus groups in the development of an item bank. Quality of Life Research, 19(1), 111–123.

    Article  PubMed  Google Scholar 

  6. Anatchkova, M. D., Ware, J. E., & Bjorner, J. B. (2011). Assessing the factor structure of a role functioning item bank. Quality of Life Research, 20, 745–758.

    Google Scholar 

  7. Gandek, B., Sinclair, S. J., Jette, A. M., & Ware, J. E., Jr. (2007). Development and initial psychometric evaluation of the participation measure for post-acute care (PM-PAC). American Journal of Physical Medicine & Rehabilitation, 86(0894-9115; 1), 57–71.

    Google Scholar 

  8. Haley, S. M., Gandek, B., Siebens, H., et al. (2008). Computerized adaptive testing for follow-up after discharge from inpatient rehabilitation: II. Participation outcomes. Archives of Physical Medicine and Rehabilitation, 89(2), 275–283.

    Google Scholar 

  9. Mulcahey, M. J., Haley, S. M., Duffy, T., Pengsheng, N., & Betz, R. R. (2008). Measuring physical functioning in children with spinal impairments with computerized adaptive testing. Journal of Pediatric Orthopaedics, 28(0271-6798; 3), 330–335.

    Google Scholar 

  10. Wilkie, D. J., Judge, M. K., Berry, D. L., Dell, J., Zong, S., & Gilespie, R. (2003). Usability of a computerized PAINReportIt in the general public with pain and people with cancer pain. Journal of Pain and Symptom Management, 25(0885-3924; 3), 213–224.

    Google Scholar 

  11. Bjorner, J. B., Kosinski, M., & Ware, J. E., Jr. (2003). Calibration of an item pool for assessing the burden of headaches: An application of item response theory to the headache impact test (HIT). Quality of Life Research, 12(8), 913–933.

    Article  PubMed  Google Scholar 

  12. Bayliss, M. S., Dewey, J. E., Dunlap, I., et al. (2003). A study of the feasibility of internet administration of a computerized health survey: The headache impact test (HIT). Quality of Life Research, 12(8), 953–961.

    Article  PubMed  CAS  Google Scholar 

  13. Ware, J. E., Jr., & Dewey, J. (2000). How to score version two of the SF-36 health survey. Lincoln, RI: QualityMetric Incorporated.

    Google Scholar 

  14. Ramsay, J. (1995). TestGraf—A program for the graphical analysis of multiple choice test and questionnaire data. Montreal, Canada: McGill University.

    Google Scholar 

  15. Teresi, J. A. (2006). Different approaches to differential item functioning in health applications. Advantages, disadvantages and some neglected topics. Medical Care, 44(11 Suppl 3), S152–S170.

    Google Scholar 

  16. Teresi, J. A., & Fleishman, J. A. (2007). Differential item functioning and health assessment. Quality of Life Research, 16(Suppl 1), 33–42.

    Article  PubMed  Google Scholar 

  17. Crane, P. K., Gibbons, L. E., Ocepek-Welikson, K., et al. (2007). A comparison of three sets of criteria for determining the presence of differential item functioning using ordinal logistic regression. Quality of Life Research, 16(Suppl 1), 69–84.

    Article  PubMed  Google Scholar 

  18. Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and likert-type (Ordinal) item scores. Ottawa, Ontario: Directorate of Human Resources Research and Evaluation, Department of National Defense.

    Google Scholar 

  19. Nagelkerke, N. J. (1991). A note on a general definition of the coefficient of determination. Biometrika, 78, 691–692.

    Article  Google Scholar 

  20. Muraki, E. (1997). Generalized partial credit model. In V. D. Linden & R. K. Hambleton (Eds.), Handbook of item response theory (pp. 153–164). New York, NY: Springer.

  21. Muraki, E., & Bock, R. (1996). Parscale—IRT based test scoring and item analysis for graded open-ended exercises and performance tasks. Chicago, IL: Scientific Software, Inc.

    Google Scholar 

  22. Edelen, M. O., & Reeve, B. B. (2007). Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Quality of Life Research, 16(Suppl 1), 5–18.

    Article  PubMed  Google Scholar 

  23. Bjorner, J. B., Smith, K., Stone, C., & Sun. X. (2007). IRTFIT: A macro for item fit and local dependence tests under IRT models.

  24. Orlando, M., & Thissen, D. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24(1), 50–64.

    Article  Google Scholar 

  25. Orlando, M., & Thissen, D. (2003). Further examination of the performance of S-X2, an item fit index for dichotomous item response theory models. Applied Psychological Measurement, 27(4), 289–298.

    Article  Google Scholar 

  26. Warm, T. A. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54, 427–450.

    Article  Google Scholar 

  27. Stewart, A. L., Greenfield, S., Hays, R. D., et al. (1989). Functional status and well-being of patients with chronic conditions. Results from the medical outcomes study. JAMA, 262(7), 907–913.

    Google Scholar 

  28. Sprangers, M. A., de Regt, E. B., Andries, F., et al. (2000). Which chronic conditions are associated with better or poorer quality of life? Journal of Clinical Epidemiology, 53(9), 895–907.

    Article  PubMed  CAS  Google Scholar 

  29. Kempen, G. I., Sanderman, R., Miedema, I., Meyboom-de Jong, B., & Ormel, J. (2000). Functional decline after congestive heart failure and acute myocardial infarction and the impact of psychological attributes. A prospective study. Quality of Life Research, 9(4), 439–450.

    Google Scholar 

  30. McHorney, C. A., Ware, J. E., Jr., & Raczek, A. E. (1993). The MOS 36-item short-form health survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Medical Care, 31, 247–263.

    Article  PubMed  CAS  Google Scholar 

  31. Hahn, E. A., Cella, D., Bode, R. K., & Hanrahan, R. T. (2010). Measuring social well-being in people with chronic illness. Social Indicators Research, 96(3), 381–401.

    Article  Google Scholar 

  32. Hahn, E. A., Devellis, R. F., Bode, R. K., et al. (2010). Measuring social health in the patient-reported outcomes measurement information system (PROMIS): Item bank development and testing. Quality of Life Research, 19(7), 1035–1044.

    Article  PubMed  Google Scholar 

  33. Reeve, B. B., Hays, R. D., Bjorner, J. B., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the patient-reported outcomes measurement information system (PROMIS). Medical Care, 45(5), S22–S31.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

The project described was supported by Award Number K01AG028760 from the National Institute on Aging. Partial salary support for Dr. Anatchkova is provided by the National Institutes of Health grant 1U01HL105268-01. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Aging or the National Institutes of Health.

Author information

Authors and Affiliations

  1. Department of Quantitative Health Sciences, Medical School, University of Massachusetts, 55 Lake Avenue North, Worcester, MA, USA

    Milena D. Anatchkova, Matthias Rose & John E. Ware Jr.

  2. Department of Psychosomatic Medicine, University Clinic Hamburg-Eppendorf and Schön Klinik Hamburg-Eilbek, Hamburg, Germany

    Matthias Rose

  3. John Ware Research Group, Worcester, MA, USA

    John E. Ware Jr.

  4. National Institute of Occupational Health, Copenhagen, Denmark

    Jakob B. Bjorner

  5. i3 QualityMetric Inc, Lincoln, RI, USA

    Jakob B. Bjorner

Authors
  1. Milena D. Anatchkova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthias Rose
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. John E. Ware Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jakob B. Bjorner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Milena D. Anatchkova.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Anatchkova, M.D., Rose, M., Ware, J.E. et al. Development of an item bank and computer adaptive test for role functioning. Qual Life Res 21, 1625–1637 (2012). https://doi.org/10.1007/s11136-011-0076-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-011-0076-4

Keywords

  • Role function
  • IRT
  • Computer adaptive test
  • CAT