Quality of Life Research

, Volume 23, Issue 6, pp 1767–1775 | Cite as

Use of cognitive interviews in the development of the PLUS-M item bank

  • Sara J. Morgan
  • Dagmar Amtmann
  • Daniel C. Abrahamson
  • Andre J. Kajlich
  • Brian J. HafnerEmail author



Measuring constructs such as mobility with patient-reported outcomes (PROs) can enhance clinical and scientific understanding of how health conditions, like lower limb amputation, impact patients’ lives. When developing PRO questionnaires, cognitive interviews (CIs) are used to examine whether survey items are understandable, clear, and meaningful. The aim of this study was to use CIs to inform item development for the Prosthetic Limb Users Survey of Mobility (PLUS-M), a PRO that measures mobility in prosthetic limb users.


Thirty-six CIs were conducted with 30 prosthetic limb users. Each participant responded to up to 30 items from the PLUS-M candidate item set. Each item was reviewed by a minimum of five participants who differed in self-reported mobility, literacy, level of amputation, and time since amputation. Items were revised based on participant feedback, and substantially revised items were re-evaluated through additional CIs.


Feedback from CIs identified substantial issues in 76 of the total 156 items. These items were subsequently modified or eliminated.


Cognitive interviews were an essential qualitative step in the development of the PLUS-M item bank and resulted in better functioning items.


Cognitive interviews Artificial limb Qualitative research Mobility Patient-reported outcome measure 



This research is supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NIH Grant Number HD-065340). The authors wish to thank Rana Salem, MS, for performing the descriptive data analysis, Silvia Christian, BA, for performing the Lexile® analysis, and Meighan Rasley, BA, for scheduling participant interviews.


  1. 1.
    Sansam, K., O’Connor, R., Neumann, V., & Bhakta, B. (2012). Can simple clinical tests predict walking ability after prosthetic rehabilitation? Journal of Rehabilitation Medicine, 44(11), 968–974. doi: 10.2340/16501977-1046.PubMedCrossRefGoogle Scholar
  2. 2.
    Lin, S. J., & Bose, N. H. (2008). Six-minute walk test in persons with transtibial amputation. Archives of Physical Medicine and Rehabilitation, 89(12), 2354–2359. doi: 10.1016/j.apmr.2008.05.021.PubMedCrossRefGoogle Scholar
  3. 3.
    Deathe, A. B., & Miller, W. C. (2005). The L test of functional mobility: Measurement properties of a modified version of the timed “up & go” test designed for people with lower-limb amputations. Physical Therapy, 85(7), 626–635.PubMedGoogle Scholar
  4. 4.
    Gailey, R. S., Roach, K. E., Applegate, E. B., Cho, B., Cunniffe, B., Licht, S., et al. (2002). The amputee mobility predictor: An instrument to assess determinants of the lower-limb amputee’s ability to ambulate. Archives of Physical Medicine and Rehabilitation, 83(5), 613–627.PubMedCrossRefGoogle Scholar
  5. 5.
    Brooks, D., Parsons, J., Hunter, J. P., Devlin, M., & Walker, J. (2001). The 2-minute walk test as a measure of functional improvement in persons with lower limb amputation. Archives of Physical Medicine and Rehabilitation, 82(10), 1478–1483. doi: 10.1053/apmr.2001.25153.PubMedCrossRefGoogle Scholar
  6. 6.
    Datta, D., Ariyaratnam, R., & Hilton, S. (1996). Timed walking test: An all-embracing outcome measure for lower-limb amputees? Clinical Rehabilitation, 10(3), 227–232. doi: 10.1177/026921559601000307.CrossRefGoogle Scholar
  7. 7.
    Stratford, P. W., Kennedy, D., Pagura, S. M. C., & Gollish, J. D. (2003). The relationship between self-report and performance-related measures: Questioning the content validity of timed tests. Arthritis and Rheumatism, 49(4), 535–540. doi: 10.1002/art.11196.PubMedCrossRefGoogle Scholar
  8. 8.
    Amtmann, D., Cook, K. F., Johnson, K. L., & Cella, D. (2011). The PROMIS initiative: Involvement of rehabilitation stakeholders in development and examples of applications in rehabilitation research. Archives of Physical Medicine and Rehabilitation, 92(10 Suppl), S12–S19. doi: 10.1016/j.apmr.2011.04.025.PubMedCentralPubMedCrossRefGoogle Scholar
  9. 9.
    Condie, E., Scott, H., & Treweek, S. (2006). Lower limb prosthetic outcome measures: A review of the literature 1995 to 2005. Journal of Prosthetics and Orthotics, 18(6), P13.CrossRefGoogle Scholar
  10. 10.
    Willis, G. B. (1999). Cognitive interviewing: A “how to” guide. Course manual prepared for a short course at the 1999 Meeting of the American Statistical Association: Research Triangle Institute.Google Scholar
  11. 11.
    Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 12(3), 229–238.CrossRefGoogle Scholar
  12. 12.
    Jobe, J. B. (2003). Cognitive psychology and self-reports: Models and methods. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 12(3), 219–227.CrossRefGoogle Scholar
  13. 13.
    Presser, S., Couper, M. P., Lessler, J. T., Martin, E., Martin, J., Rothgeb, J. M., et al. (2004). Methods for testing and evaluating survey questions. Public Opinion Quarterly, 68(1), 109–130.CrossRefGoogle Scholar
  14. 14.
    Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks: Sage. Incorporated.Google Scholar
  15. 15.
    DeWalt, D. A., Rothrock, N., Yount, S., & Stone, A. A. (2007). Evaluation of item candidates: The PROMIS qualitative item review. Medical Care, 45(5 Suppl 1), S12–S21. doi: 10.1097/01.mlr.0000254567.79743.e2.PubMedCentralPubMedCrossRefGoogle Scholar
  16. 16.
    Beatty, P. C., & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71(2), 287–311. doi: 10.1093/poq/nfm006.CrossRefGoogle Scholar
  17. 17.
    Gershon, R. C., Lai, J. S., Bode, R., Choi, S., Moy, C., Bleck, T., et al. (2012). Neuro-QOL: Quality of life item banks for adults with neurological disorders: Item development and calibrations based upon clinical and general population testing. Quality of Life Research, 21(3), 475–486. doi: 10.1007/s11136-011-9958-8.PubMedCentralPubMedCrossRefGoogle Scholar
  18. 18.
    Cella, D., Nowinski, C., Peterman, A., Victorson, D., Miller, D., Lai, J.-S., et al. (2011). The neurology quality-of-life measurement initiative. Archives of Physical Medicine and Rehabilitation, 92(10 Suppl), S28–S36. doi: 10.1016/j.apmr.2011.01.025.PubMedCentralPubMedCrossRefGoogle Scholar
  19. 19.
    Patient Reported Outcomes Measurement Information System (PROMIS®). (2012). PROMIS® instrument development and psychometric evaluation scientific standards. Accessed 19 Mar 2013.Google Scholar
  20. 20.
    Franchignoni, F., Giordano, A., Ferriero, G., Orlandini, D., Amoresano, A., & Perucca, L. (2007). Measuring mobility in people with lower limb amputation: Rasch analysis of the mobility section of the prosthesis evaluation questionnaire. Journal of Rehabilitation Medicine, 39(2), 138–144. doi: 10.2340/16501977-0033.PubMedCrossRefGoogle Scholar
  21. 21.
    Ziegler-Graham, K., MacKenzie, E. J., Ephraim, P. L., Travison, T. G., & Brookmeyer, R. (2008). Estimating the prevalence of limb loss in the United States: 2005 to 2050. Archives of Physical Medicine and Rehabilitation, 89(3), 422–429. doi: 10.1016/j.apmr.2007.11.005.PubMedCrossRefGoogle Scholar
  22. 22.
    Ephraim, P. L., Dillingham, T. R., Sector, M., Pezzin, L. E., & Mackenzie, E. J. (2003). Epidemiology of limb loss and congenital limb deficiency: A review of the literature. Archives of Physical Medicine and Rehabilitation, 84(5), 747–761.PubMedCrossRefGoogle Scholar
  23. 23.
    Krosnick, J. A., Holbrook, A. L., Berent, M. K., Carson, R. T., Hanemann, W. M., Kopp, R. J., et al. (2002). The impact of “no opinion” response options on data quality: Non-attitude reduction or an invitation to satisfice? Public Opinion Quarterly, 66(3), 371–403. doi: 10.1086/341394.CrossRefGoogle Scholar
  24. 24.
    Stenner, A. J., Horablin, I., Smith, D. R., & Smith, M. (1988). The Lexile framework. Durham, NC: MetaMetrics. Inc.Google Scholar
  25. 25.
    US Department of Health and Human Services (USDHHS). (2009). Guidance for industry. Patient-reported outcome measures: Use in medical product development to support labeling claims (US FDA guidance for industry related to the development and review of PRO measures). Retrieved from Information/Guidances/UCM193282.pdf. Accessed 1 Dec 2010.
  26. 26.
    Patrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E., et al. (2011). Content validity—Establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 2—Assessing respondent understanding. Value in Health, 14(8), 978–988. doi: 10.1016/j.jval.2011.06.013.PubMedCrossRefGoogle Scholar
  27. 27.
    Wilkerson, G. S., & Robertson, G. J. (2006). WRAT4 wide range achievement test professional manual. Lutz, FL: Psychological Assessment Resources Inc.Google Scholar
  28. 28.
    Davis, T. C., Long, S. W., Jackson, R. H., Mayeaux, E. J., George, R. B., Murphy, P. W., et al. (1993). Rapid estimate of adult literacy in medicine: A shortened screening instrument. Family Medicine, 25(6), 391–395.PubMedGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2014

Authors and Affiliations

  • Sara J. Morgan
    • 1
  • Dagmar Amtmann
    • 1
  • Daniel C. Abrahamson
    • 1
  • Andre J. Kajlich
    • 1
  • Brian J. Hafner
    • 1
    Email author
  1. 1.Department of Rehabilitation MedicineUniversity of WashingtonSeattleUSA

Personalised recommendations