Abstract
Objective: To evaluate the feasibility of computerized adaptive testing (CAT) and the reliability and validity of CAT-based estimates of headache impact scores in comparison with 'static' surveys. Methods: Responses to the 54-item Headache Impact Test (HIT) were re-analyzed for recent headache sufferers (n = 1016) who completed telephone interviews during the National Survey of Headache Impact (NSHI). Item response theory (IRT) calibrations and the computerized dynamic health assessment (DYNHA®) software were used to simulate CAT assessments by selecting the most informative items for each person and estimating impact scores according to pre-set precision standards (CAT-HIT). Results were compared with IRT estimates based on all items (total-HIT), computerized 6-item dynamic estimates (CAT-HIT-6), and a developmental version of a 'static' 6-item form (HIT-6-D). Analyses focused on: respondent burden (survey length and administration time), score distributions ('ceiling' and 'floor' effects), reliability and standard errors, and clinical validity (diagnosis, level of severity). A random sample (n = 245) was re-assessed to test responsiveness. A second study (n = 1103) compared actual CAT surveys and an improved 'static' HIT-6 among current headache sufferers sampled on the Internet. Respondents completed measures from the first study and the generic SF-8™ Health Survey; some (n = 540) were re-tested on the Internet after 2 weeks. Results: In the first study, simulated CAT-HIT and total-HIT scores were highly correlated (r = 0.92) without 'ceiling' or 'floor' effects and with a substantial reduction (90.8%) in respondent burden. Six of the 54 items accounted for the great majority of item administrations (3603/5028, 77.6%). CAT-HIT reliability estimates were very high (0.975–0.992) in the range where 95% of respondents scored, and relative validity (RV) coefficients were high for diagnosis (RV = 0.87) and severity (RV = 0.89); patient-level classifications were accurate 91.3% for a diagnosis of migraine. For all three criteria of change, CAT-HIT scores were more responsive than all other measures. In the second study, estimates of respondent burden, item usage, reliability and clinical validity were replicated. The test–retest reliability of CAT-HIT was 0.79 and alternate forms coefficients ranged from 0.85 to 0.91. All correlations with the generic SF-8 were negative. Conclusions: CAT-based administrations of headache impact items achieved very large reductions in respondent burden without compromising validity for purposes of patient screening or monitoring changes in headache impact over time. IRT models and CAT-based dynamic health assessments warrant testing among patients with other conditions.
Similar content being viewed by others
References
Hambleton RK, Swaminathan H. Item Response Theory: Principles and Applications. Boston, MA: Kluwer-Nijhoff, 1985.
Thissen D. Reliability and measurement precision. In: Wainer H, et al. (eds), Computerized Adaptive Testing: A Primer, 2nd ed. Mahwah, NJ: Lawrence Erlbaum Associates 2000: 159–183.
Wainer H. Introduction and history. In: Wainer H, Dorans NJ, Flaugher R, et al. (eds), Computerized Adaptive Testing: A Primer. 2nd ed. Hillsdale (NJ): Lawrence Erlbaum Associates, 2000.
Lipton RB, Scher AI, Kolodner K, et al. Migraine in the United States: Epidemiology and patterns of health care use. Neurology 2002; 58(6): 885–894.
Ware JE, Bjorner JB, Kosinski M. Practical implications of item response theory and computerized adaptive testing: A brief summary of ongoing studies of widely used headache impact scales. Med Care 2000; 38(9)(Suppl 2): II73–II82.
Lipton RB, Diamond S, Reed M, et al. Migraine diagnosis and treatment: Results from the American Migraine Study II. Headache 2001; 41(7): 638–645.
Bjorner J, Kosinski M, Ware JE. Calibration of an item pool for assessing the burden of headaches: An application of item response theory to the Headache Impact Test (HITTM). Qual Life Res 2003; 12: 913–933.
Thissen D, Wainer H. Test Scoring. Mahwah, NJ: Lawrence Erlbaum Associates, 2001.
Sands WA, Waters BK, McBride JR. Computerized Adaptive Testing: From Inquiry to Operation.Washington, DC: American Psychological Association, 1997.
Stewart WF, Lipton RB, Liberman J. Variation in migraine prevalence by race. Neurology 1996; 47: 52–59.
Kosinski M, Bayliss MS, Bjorner JB, et al. A six-item short-form survey for measuring headache impact: The HIT-6TM. Qual Life Res 2003; 12: 963–974.
Ware JE, Kosinski M, Dewey JE, et al. How to Score and Interpret Single-item Health Status Measures: A Manual for Users of the SF-8 Health Survey. Lincoln, RI: Quality-Metric Incorporated, 2001.
Jhingram P, Osterhaus JT, Miller DW, et al. Development and validation of the Migraine Specific Quality of Life Questionnaire. Headache 1998; 38(4): 295–302.
Jacobson GP, Ranadar NM, Norris L, et al. Headache Disability Inventory (HDI): Short term test-retest reliability and spouse perceptions. Headache 1995; 35(9): 534–539.
Stewart WF, Lipton RB, Simon D, et al. Validity of an illness severity measure for headache in a population of migraine sufferers. Pain 1999; 291–304.
Stewart WF, Lipton RB, Kolonder K, et al. Reliability of the Migraine Disability Assessment Score in a population-based sample of headache sufferers. Cephalalgia 1999; 19: 107–114.
Wainer H, Dorans NJ, Flaugher R, et al. Computerized Adaptive Testing: A Primer. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates, 2000.
Bjorner J, Ware JE. Using modern psychometric methods to measure health outcomes. Med Outcomes Trust Monitor 1998; 3(2): 11–16.
McBride JR. Research antecedents of applied adaptive testing. In: Sands WA, Waters BK, McBride JR (eds), Computerized Adaptive Testing: From Inquiry to Operation. Washington, DC: American Psychological Association, 1997.
Davies AR, Kram N (eds). Monitoring Health Outcomes Study Guide: Advanced Topics. Woodbridge, NJ: Health-Stat Productions, Incorporated, 2000.
StataCorp. Stata Statistical Software: Release 7.0. College Station, TX: Stata Corporation 2001.
Aday LA. Designing and Conducting Health Surveys. San Francisco, CA: Jossey-Bass Publishers, 1996.
Kerlinger FN. Foundations of Behavioral Research. 2nd ed. New York, NY: Holt, Rinehart and Winston, Inc., 1973.
Stewart AL, Ware JE (eds), Measuring Functioning and Well-being: The Medical Outcomes Study Approach. Raleigh-Durham, NC: Duke University Press, 1992.
Bayliss MS, Dewey JE, Dunlap I, et al. A study of the feasibility of Internet administration of a computerized health survey: The Headache Impact Test (HITTM). Qual Life Res 2003; 12: 953–961.
Becker WJ, Ware JE. What does it mean to have migraine? Neurology 2000; 55(5): 610–611.
Hetter RD, Segall FDO, Bloxom BM. Evaluating item calibration medium in computerized adaptive testing. In: Sands, et al. (eds), Computerized Adaptive Testing: From Inquiry to Operation, Washington, DC: American Psychological Association, 1997; 161–167.
Patrick DL, Chiang YP, (eds). Health Outcomes Methodology: Symposium Proceedings. Med Care 2000; 38(9)(Suppl 2): II1–II210.
Lerner D, Amick BC, Rogers WH, et al. The Work Limitations Questionnaire. Med Care 2001; 39(1): 72–85.
Dodd BG, Fitzpatrick SJ. Alternatives for scoring CBTs. In: Mills et al. (eds), Computer-Based Testing: Building the Foundation for Future Assessments. Mahwah, NJ: Lawrence Erlbaum Associates, 2002.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Ware Jr, J.E., Kosinski, M., Bjorner, J.B. et al. Applications of computerized adaptive testing (CAT) to the assessment of headache impact. Qual Life Res 12, 935–952 (2003). https://doi.org/10.1023/A:1026115230284
Issue Date:
DOI: https://doi.org/10.1023/A:1026115230284