Efficiency of static and computer adaptive short forms compared to full-length measures of depressive symptoms
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.Get Access
Short-form patient-reported outcome measures are popular because they minimize patient burden. We assessed the efficiency of static short forms and computer adaptive testing (CAT) using data from the Patient-Reported Outcomes Measurement Information System (PROMIS) project.
We evaluated the 28-item PROMIS depressive symptoms bank. We used post hoc simulations based on the PROMIS calibration sample to compare several short-form selection strategies and the PROMIS CAT to the total item bank score.
Compared with full-bank scores, all short forms and CAT produced highly correlated scores, but CAT outperformed each static short form in almost all criteria. However, short-form selection strategies performed only marginally worse than CAT. The performance gap observed in static forms was reduced by using a two-stage branching test format.
Using several polytomous items in a calibrated unidimensional bank to measure depressive symptoms yielded a CAT that provided marginally superior efficiency compared to static short forms. The efficiency of a two-stage semi-adaptive testing strategy was so close to CAT that it warrants further consideration and study.
- Bjorner, JB, Chang, CH, Thissen, D, Reeve, BB (2007) Developing tailored instruments: item banking and computerized adaptive assessment. Quality of Life Research 16: pp. 95-108 CrossRef
- Thissen, D, Reeve, BB, Bjorner, JB, Chang, CH (2007) Methodological issues for building item banks and computerized adaptive scales. Quality of Life Research 16: pp. 109-119 CrossRef
- Cella, D, Yount, S, Rothrock, N, Gershon, R, Cook, K, Reeve, B, Ader, D, Fries, JF, Bruce, B, Rose, M (2007) The patient-reported outcomes measurement information system (PROMIS): progress of an NIH roadmap cooperative group during its first two years. Medical Care 45: pp. S3-S11 CrossRef
- Belov, DI, Armstrong, RD (2008) A Monte Carlo approach to the design, assembly, and evaluation of multistage adaptive tests. Applied Psychological Measurement 32: pp. 119-137 CrossRef
- Pilkonis, P. A., Choi, S. W., Reise, S. P., Stover, A. M., Riley, W. T., & Cella, D. (in preparation). The development of scales for emotional distress from the patient-reported outcomes measurement information system (PROMIS): Depression, Anxiety, and Anger.
- Fliege, H, Becker, J, Walter, O, Bjorner, J, Klapp, B, Rose, M (2005) Development of a computer-adaptive test for depression (D-CAT). Quality of Life Research 14: pp. 2277-2291 CrossRef
- Gardner, W, Shear, K, Kelleher, K, Pajer, K, Mammen, O, Buysse, D, Frank, E (2004) Computerized adaptive measurement of depression: A simulation study. BMC Psychiatry 4: pp. 13 CrossRef
- Gibbons, RD, Weiss, DJ, Kupfer, DJ, Frank, E, Fagiolini, A, Grochocinski, VJ, Bhaumik, DK, Stover, A, Bock, RD, Immekus, JC (2008) Using computerized adaptive testing to reduce the burden of mental health assessment. Psychiatric Services 59: pp. 361-368 CrossRef
- Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph, 17.
- Thissen, D, Chen, W-H, Bock, RD (2003) Multilog (version 7) [Computer software]. Scientific Software International, Lincolnwood, IL
- Kang, T, Chen, T (2008) Performance of the generalized S-X2 item fit index for polytomous IRT models. Journal of Educational Measurement 45: pp. 391-406 CrossRef
- Orlando, M, Thissen, D (2003) Further investigation of the performance of S-X2: An item fit index for use with dichotomous item response theory models. Applied Psychological Measurement 27: pp. 289-298 CrossRef
- Bjorner, JB, Smith, KJ, Orlando, M, Stone, C, Thissen, D, Sun, X (2006) IRTFIT: A macro for item fit and local dependence tests under IRT models. Quality Metric, Inc, Lincoln, RI
- Liu, H., Cella, D., Gershon, R., Shen, J., Morales, L. S., Riley, W. T., & Hays, R. D. (in press). Representativeness of the PROMIS Internet Panel. Journal of Clinical Epidemiology.
- Muthen, L. K. & Muthen, B. O. (1998). Mplus user’s guide.
- Choi, SW (2009) Firestar: Computerized adaptive testing simulation program for polytomous IRT models. Applied Psychological Measurement 33: pp. 644-645 CrossRef
- Lord, FM (1980) Applications of item response theory to practical testing problems. Erlbaum, Hillsdale, NJ
- Weiss, DJ (1982) Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement 6: pp. 473-492 CrossRef
- Chang, H-H, Ying, Z (1996) A global information approach to computerized adaptive testing. Applied Psychological Measurement 20: pp. 213-229 CrossRef
- Lima Passos, V, Berger, MPF, Tan, FE (2007) Test design optimization in CAT early stage with the nominal response model. Applied Psychological Measurement 31: pp. 213-232 CrossRef
- Linden, WJ, Pashley, PJ Item selection and ability estimator in adaptive testing. In: Linden, WJ, Glas, CAW eds. (2000) Computerized adaptive testing: Theory and practice. Kluwer Academic, Boston, MA, pp. 1-25
- Veerkamp, WJJ, Berger, MPF (1997) Some new item selection criteria for adaptive testing. Journal of Educational and Behavioral Statistics 22: pp. 203-226
- Bock, RD, Mislevy, RJ (1982) Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement 6: pp. 431-444 CrossRef
- Choi, SW, Swartz, RJ (2009) Comparison of CAT item selection criteria for polytomous items. Applied Psychological Measurement 33: pp. 419-440 CrossRef
- Linden, W (1998) Optimal assembly of psychological and education tests. Applied Psychological Measurement 22: pp. 195-211 CrossRef
- Reise, SP, Henson, JM (2000) Computerization and adaptive administration of the NEO PI-R. Assessment 7: pp. 347-364 CrossRef
- Hol, AM, Vorst, HCM, Mellenbergh, GJ (2007) Computerized adaptive testing for polytomous motivation items: administration mode effects and a comparison with short forms. Applied Psychological Measurement 31: pp. 412-429 CrossRef
- Kendall, MG, Babington, SB (1939) The problem of m rankings. The Annals of Mathematical Statistics 10: pp. 275-287 CrossRef
- Cohen, J (1960) A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20: pp. 37-46 CrossRef
- Yule, GU (1912) On the methods of measuring association between two attributes. Journal of the Royal Statistical Society 75: pp. 579-652 CrossRef
- Warrens, M (2008) On association coefficients for 2 × 2 tables and properties that do not depend on the marginal distributions. Psychometrika 73: pp. 777-789 CrossRef
- Altman, DG, Bland, JM (1994) Diagnostic tests 2: Predictive values. British Journal of Medicine 309: pp. 102
- Strauss, ME, Smith, GT (2009) Construct validity: advances in theory and methodology. Annual Review of Clinical Psychology 5: pp. 1-25 CrossRef
- Reise, SP, Morizot, J, Hays, RD (2007) The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Quality of Life Research 16: pp. 19-31 CrossRef
- Dodd, BG, Koch, WR, Ayala, RJ (1989) Operational characteristics of adaptive testing procedures using the graded response model. Applied Psychological Measurement 13: pp. 129-143 CrossRef
- Smith, GT, McCarthy, DM, Anderson, KG (2000) On the sins of short-form development. Psychological Assessment 12: pp. 102-111 CrossRef
- Efficiency of static and computer adaptive short forms compared to full-length measures of depressive symptoms
Quality of Life Research
Volume 19, Issue 1 , pp 125-136
- Cover Date
- Print ISSN
- Online ISSN
- Springer Netherlands
- Additional Links
- Computer adaptive testing
- Item response theory
- Short form
- Two-stage testing
- Industry Sectors
- Author Affiliations
- 1. Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, 710 N. Lake Shore Dr, Chicago, IL, 60611, USA
- 2. Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
- 3. Department of Psychiatry, University of Pittsburgh Medical Center, Pittsburgh, PA, USA
- 4. Department of Medicine, University of California, Los Angeles, Los Angeles, CA, USA
- 5. Health Program, RAND, Santa Monica, CA, USA