Validation of PROMIS® Physical Function Computerized Adaptive Tests for Orthopaedic Foot and Ankle Outcome Research
- 1k Downloads
In 2012, the American Orthopaedic Foot & Ankle Society® established a national network for collecting and sharing data on treatment outcomes and improving patient care. One of the network’s initiatives is to explore the use of computerized adaptive tests (CATs) for patient-level outcome reporting.
We determined whether the CAT from the NIH Patient Reported Outcome Measurement Information System® (PROMIS®) Physical Function (PF) item bank provides efficient, reliable, valid, precise, and adequately covered point estimates of patients’ physical function.
After informed consent, 288 patients with a mean age of 51 years (range, 18–81 years) undergoing surgery for common foot and ankle problems completed a web-based questionnaire. Efficiency was determined by time for test administration. Reliability was assessed with person and item reliability estimates. Validity evaluation included content validity from expert review and construct validity measured against the PROMIS® Pain CAT and patient responses based on tradeoff perceptions. Precision was assessed by standard error of measurement (SEM) across patients’ physical function levels. Instrument coverage was based on a person-item map.
Average time of test administration was 47 seconds. Reliability was 0.96 for person and 0.99 for item. Construct validity against the Pain CAT had an r value of −0.657 (p < 0.001). Precision had an SEM of less than 3.3 (equivalent to a Cronbach’s alpha of ≥ 0.90) across a broad range of function. Concerning coverage, the ceiling effect was 0.32% and there was no floor effect.
The PROMIS® PF CAT appears to be an excellent method for measuring outcomes for patients with foot and ankle surgery. Further validation of the PROMIS® item banks may ultimately provide a valid and reliable tool for measuring patient-reported outcomes after injuries and treatment.
Level of Evidence
Level III, diagnostic study. See Instructions for Authors for a complete description of levels of evidence.
KeywordsItem Response Theory Item Bank Computerize Adaptive Test Foot Function Index Hallux Rigidus
The authors thank Ian Alexander MD (Cleveland Clinic–Medina Hospital, Medina, OH, USA) for his extensive contributions to the AOFAS OFAR Network and this project. We acknowledge the contributing investigators of the OFAR Network, including Timothy R. Daniels MD (University of Toronto, Toronto, ON, Canada), James Brodsky MD (Baylor University, Houston, TX, USA), Jon Deland MD (Hospital for Special Surgery, New York, NY, USA), Scott Ellis MD (Hospital for Special Surgery, New York, NY, USA), Phinit Phisitkul MD (University of Iowa, Iowa City, IA, USA), W. Hodges Davis MD (OrthoCarolina Research Institute, Charlotte, NC, USA), Susan Ishikawa MD (The University of Tennessee–Campbell Clinic, Memphis, TN, USA), Loretta B. Chou MD (Stanford University, Redwood City, CA, USA), John Ketz MD (University of Rochester, Rochester, NY, USA), Margaret Chilvers MD (University of Arizona, Tucson, AZ, USA), Sam Flemister MD (University of Rochester, Rochester, NY, USA), Florian Nickisch MD (University of Utah, Salt Lake City, UT, USA), and Timothy Beals MD (University of Utah, Salt Lake City, UT, USA). We are thankful for the contributions of each of the 10 clinical testing sites in the OFAR Network. We also thank Lousanne Lofgren BS (AOFAS, Rosemont, IL, USA) and Rose Olea BS (AOFAS, Rosemont, IL, USA) for their tireless work on this project. Special thanks go to Christine Cheng BS (University of Utah, Salt Lake City, UT, USA) and Shirley D. Hon BS (University of Utah, Salt Lake City, UT, USA) for their tremendous support in data collection, analysis, manuscript preparation, and review. Finally, the study design, statistical and psychometric support, and computing resources provided by the Center for Outcomes Research and Assessment at the University of Utah Department of Orthopaedics are gratefully acknowledged.
- 3.Bajaj JS, Thacker LR, Wade JB, Sanyal AJ, Heuman DM, Sterling RK, Gibson DP, Stravitz RT, Puri P, Fuchs M, Luketic V, Noble N, White M, Bell D, Revicki DA. PROMIS computerised adaptive tests are dynamic instruments to measure health-related quality of life in patients with cirrhosis. Aliment Pharmacol Ther. 2011;34:1123–1132.PubMedCrossRefGoogle Scholar
- 8.Bond TG, Fox CM. Applying the Rasch Model: Fundamental Measurement in the Human Sciences. Mahwah, NJ: Lawrence Erlbaum; 2001.Google Scholar
- 11.Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, Ader D, Fries JF, Bruce B, Rose M; PROMIS Cooperative Group. The Patient Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care. 2007;45:S3–S11.PubMedCrossRefGoogle Scholar
- 12.Davidshofer KR, Murphy CO. Psychological Testing: Principles and Applications. 6th ed. Upper Saddle River, NJ: Pearson/Prentice Hall; 2005.Google Scholar
- 21.Kocher MS, Horan MP, Briggs KK, Richardson TR, O’Holleran J, Hawkins RJ. Reliability, validity, and responsiveness of the American Shoulder and Elbow Surgeons Subjective Shoulder Scale in patients with shoulder instability, rotator cuff disease, and glenohumeral arthritis. J Bone Joint Surg Am. 2005;87:2006–2011.PubMedCrossRefGoogle Scholar
- 27.Martin DP, Engelberg R, Agel J, Swiontkowski MF. Comparison of the Musculoskeletal Function Assessment questionnaire with the Short Form-36, the Western Ontario and McMaster Universities Osteoarthritis Index, and the Sickness Impact Profile health-status measures. J Bone Joint Surg Am. 1997;79:1323–1335.PubMedGoogle Scholar
- 33.Rasch G. Probabilistic Models for Some Intelligence and Attainment Tests. Chicago, IL: The University of Chicago Press; 1980.Google Scholar
- 41.Suk M, Hanson B, Norvell D, Helfet DL. AO Handbook of Musculoskeletal Outcomes Measures and Instruments. New York, NY: Thieme; 2009.Google Scholar
- 42.Tuttleman M, Pillemer SR, Tilley BC, Fowler SE, Buckley LM, Alaracon GS, Trentham DE, Neuner R, Clegg DO, Leisen JC, Heyse SP. A cross sectional assessment of health status instruments in patients with rheumatoid arthritis participating in a clinical trial. J Rheumatol. 1997;24:1910–1915.PubMedGoogle Scholar
- 43.Wright BD, Masters GN. Rating Scale Analysis: Rasch Measurement. Chicago, IL: MESA Press; 1982.Google Scholar