Applied Research in Quality of Life

, Volume 9, Issue 3, pp 667–682 | Cite as

Evaluating the Face Validity of the ICECAP-O Capabilities Measure: A “Think Aloud” Study with Hip and Knee Arthroplasty Patients

  • Jeremy HorwoodEmail author
  • Eileen Sutton
  • Joanna Coast


The ICECAP-O index of capabilities measure for older people is intended to be used in the evaluation of health and social care interventions. Focusing on quality of life, rather than health or other influences on quality of life. This study evaluated the face validity of the self-administered ICECAP-O capabilities measure for older people by investigating how participants interpret and respond to questions using the cognitive interviewing technique. Twenty patients with osteoarthritis of the knee or hip participated in a cognitive interview whilst completing the ICECAP-O capabilities measure. Cognitive interviews were conducted using the concurrent ‘think aloud’ design. All interviews were audio recorded and transcribed verbatim and analysed (i) using a standardised classification scheme to identify response problems and (ii) qualitatively thematically analysed to explore the nature of the problems that the participants experienced when completing the measure. Problems were identified in 7 % of participants’ responses. The majority of problems identified were comprehension problems. Thematic analysis highlighted the extent to which participants’ distinguished between functioning and capability. Cognitive interviewing was a valuable technique for pre-testing the face validity, acceptability and content validity of the ICECAP-O capabilities measure. Participants had minimal difficulties completing the ICECAP-O capabilities measure. Those difficulties identified have prompted suggestions for improving the measure.


Cognitive interviewing Think aloud Osteoarthritis Capabilities Health outcome instrument 



The authors would like to thank those who participated in the “think aloud” interviews and the staff at the orthopaedic clinics that helped with recruitment and Teressa McIlvenna for assisting with data collection. This work was supported by the MRC Health Services Research Collaboration.


  1. Bowden, A., Fox-Rushby, J. A., Nyandieka, L., & Wanjau, J. (2002). Methods for pre-testing and piloting survey questions: illustrations from the KENQOL survey of health-related quality of life. Health Policy and Planning, 17(3), 322–330.CrossRefGoogle Scholar
  2. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.CrossRefGoogle Scholar
  3. Brazier, J., Roberts, J., & Deverill, M. (2002). The estimation of a preference-based measure of health from the SF-36. [Research Support, Non-U.S. Gov’t]. Journal of Health Economics, 21(2), 271–292.CrossRefGoogle Scholar
  4. Brooks, R. (1996). EuroQol: the current state of play. [Review]. Health Policy, 37(1), 53–72.CrossRefGoogle Scholar
  5. Coast, J., Flynn, T. N., Natarajan, L., Sproston, K., Lewis, J., Louviere, J. J., et al. (2008a). Valuing the ICECAP capability index for older people. [Research Support, Non-U.S. Gov’t]. Social Science and Medicine, 67(5), 874–882. doi: 10.1016/j.socscimed.2008.05.015.CrossRefGoogle Scholar
  6. Coast, J., Peters, T. J., Natarajan, L., Sproston, K., & Flynn, T. (2008b). An assessment of the construct validity of the descriptive system for the ICECAP capability measure for older people. [Research Support, Non-U.S. Gov’t Validation Studies]. Quality of Life Research, 17(7), 967–976. doi: 10.1007/s11136-008-9372-z.CrossRefGoogle Scholar
  7. Coast, J., Smith, R., & Lorgelly, P. (2008c). Should the capability approach be applied in health economics? Health Economics, 17(6), 667–670.CrossRefGoogle Scholar
  8. Coast, J., Smith, R. D., & Lorgelly, P. (2008d). Welfarism, extra-welfarism and capability: the spread of ideas in health economics. Social Science and Medicine, 67(7), 1190–1198.CrossRefGoogle Scholar
  9. Collins, D. (2003). Pretesting survey instruments: an overview of cognitive methods. Quality of Life Research, 12(3), 229–238.CrossRefGoogle Scholar
  10. Comans, T. A., Peel, N. M., Gray, L. C., & Scuffham, P. A. (2013). Quality of life of older frail persons receiving a post-discharge program. Health and Quality of Life Outcomes, 11, 58. doi: 10.1186/1477-7525-11-58.CrossRefGoogle Scholar
  11. Couzner, L., Ratcliffe, J., Lester, L., Flynn, T., & Crotty, M. (2012). Measuring and valuing quality of life for public health research: application of the ICECAP-O capability index in the Australian general population. International Journal of Public Health. doi: 10.1007/s00038-012-0407-4.Google Scholar
  12. Davis, J., Liu-Ambrose, T., Richardson, C., Bryan, S. (2012). A comparison of the ICECAP-O with EQ-5D in a falls prevention clinical setting: are they complements or substitutes? Quality of Life Research, 1–9. doi: 10.1007/s11136-012-0225-4.
  13. Davis, J. C., Bryan, S., McLeod, R., Rogers, J., Khan, K., & Liu-Ambrose, T. (2012b). Exploration of the association between quality of life, assessed by the EQ-5D and ICECAP-O, and falls risk, cognitive function and daily function, in older adults with mobility impairments. [Journal article]. BMC Geriatrics, 12(1), 65.CrossRefGoogle Scholar
  14. Drennan, J. (2003). Cognitive interviewing: verbal data in the design and pre-testing of questionnaires. Journal of Advanced Nursing, 42, 57–63.CrossRefGoogle Scholar
  15. Ericsson, K., & Simon, H. (1993). Protocol analysis: Verbal reports as data. Cambridge: MIT Press.Google Scholar
  16. Flynn, T., Chan, P., Coast, J., & Peters, T. (2011). Assessing quality of life among British older people using the ICEPOP CAPability (ICECAP-O) measure. Applied Health Economics and Health Policy, 9(5), 317–329. doi: 10.2165/11594150-000000000-00000.CrossRefGoogle Scholar
  17. Francis, J., & Byford, S. (2011). SCIE’s approach to economic evaluation in social care. London: Social Care Institute for Excellence.Google Scholar
  18. French, D. P., Cooke, R., McLean, N., Williams, M., & Sutton, S. (2007). What do people think about when they answer theory of planned behaviour questionnaires? A ‘think aloud’ study. Journal of Health Psychology, 12(4), 672–687.CrossRefGoogle Scholar
  19. Friedenrich, C., Courneya, K., & Bryant, H. (1998). The lifetime total physical activity questionnaire: development and reliability. Medicine and Science in Sports and Exercise, 30, 266–274.CrossRefGoogle Scholar
  20. Greenhalgh, J., Ford, H., Long, A. F., & Hurst, K. (2004). The MS Symptom and Impact Diary (MSSID): psychometric evaluation of a new instrument to measure the day to day impact of multiple sclerosis. Journal of Neurology, Neurosurgery and Psychiatry, 75(4), 577–582.CrossRefGoogle Scholar
  21. Grewal, I., Lewis, J., Flynn, T., Brown, J., Bond, J., & Coast, J. (2006). Developing attributes for a generic quality of life measure for older people: preferences or capabilities? [Research Support, Non-U.S. Gov’t]. Social Science and Medicine, 62(8), 1891–1901. doi: 10.1016/j.socscimed.2005.08.023.CrossRefGoogle Scholar
  22. Henderson, C., Knapp, M., Fernandez, J. L., Beecham, J., Hirani, S. P., Cartwright, M., . . . Whole System Demonstrator evaluation, t. (2013). Cost effectiveness of telehealth for patients with long term conditions (Whole Systems Demonstrator telehealth questionnaire study): nested economic evaluation in a pragmatic, cluster randomised controlled trial. [Research Support, Non-U.S. Gov’t]. BMJ, 346, f1035. doi: 10.1136/bmj.f1035.
  23. Jobe, J. B., & Mingay, D. J. (1989). Cognitive research improves questionnaires. American Journal of Public Health, 79(8), 1053–1055.CrossRefGoogle Scholar
  24. Kaklamanou, D., Armitage, C. J., & Jones, C. R. (2012). A further look into compensatory health beliefs: a think aloud study. [Journal article]. British Journal of Health Psychology, 24(10), 2044–8287.Google Scholar
  25. Landis, J., & Koch, G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.CrossRefGoogle Scholar
  26. Makai, P., Brouwer, W. B., Koopmanschap, M. A., & Nieboer, A. A. (2012). Capabilities and quality of life in Dutch psycho-geriatric nursing homes: an exploratory study using a proxy version of the ICECAP-O. Quality of Life Research, 21(5), 801–812. doi: 10.1007/s11136-011-9997-1.CrossRefGoogle Scholar
  27. Makai, P., Koopmanschap, M. A., Brouwer, W. B., & Nieboer, A. A. (2013). A validation of the ICECAP-O in a population of post-hospitalized older people in the Netherlands. Health and Quality of Life Outcomes, 11, 57. doi: 10.1186/1477-7525-11-57.CrossRefGoogle Scholar
  28. Mallinson, S. (2002). Listening to respondents: a qualitative assessment of the Short-Form 36 Health Status Questionnaire. Social Science & Medicine, 54(1), 11–21.CrossRefGoogle Scholar
  29. McCabe, C., Stevens, K., Roberts, J., & Brazier, J. (2005). Health state values for the HUI 2 descriptive system: results from a UK survey. [Research Support, Non-U.S. Gov’t]. Health Economics, 14(3), 231–244. doi: 10.1002/hec.925.CrossRefGoogle Scholar
  30. Mitchell, P. M., Roberts, T. E., Barton, P. M., Pollard, B. S., & Coast, J. (2013). Predicting the ICECAP-O capability index from the WOMAC osteoarthritis index: is mapping onto capability from condition-specific health status questionnaires feasible? Medical Decision Making, 33(4), 547–557. doi: 10.1177/0272989x12475092.CrossRefGoogle Scholar
  31. Murtagh, F. E., Addington-Hall, J. M., & Higginson, I. J. (2007). The value of cognitive interviewing techniques in palliative care research. Palliative Medicine, 21(2), 87–93.CrossRefGoogle Scholar
  32. National Joint Register. (2012). National Joint Register.
  33. Ratcliffe, J., Lester, L. H., Couzner, L., & Crotty, M. (2013). An assessment of the relationship between informal caring and quality of life in older community-dwelling adults–more positives than negatives? Health & Social Care in the Community, 21(1), 35–46. doi: 10.1111/j.1365-2524.2012.01085.x.CrossRefGoogle Scholar
  34. Robeyns, I. (2005). The capability approach: a theoretical survey. Journal of Human Development, 6(1), 93–117. doi: 10.1080/146498805200034266.CrossRefGoogle Scholar
  35. Robeyns, I. (2006). The capability approach in practice*. Journal of Political Philosophy, 14(3), 351–376. doi: 10.1111/j.1467-9760.2006.00263.x.CrossRefGoogle Scholar
  36. Rosal, M. C., Carbone, E. T., & Goins, K. V. (2003). Use of cognitive interviewing to adapt measurement instruments for low-literate Hispanics. The Diabetes Educator, 29(6), 1006–1017.CrossRefGoogle Scholar
  37. Sen, A. (1992). Inequality reexamined. New York: Russell Sage.Google Scholar
  38. Sen, A. (Ed.). (1993). Capability and well-being. Oxford: Clarendon.Google Scholar
  39. Tourangeau, R. (1984). Cognitive sciences and survey methods. In T. Jabine, M. Straf, J. Tanur, & R. Tourangeau (Eds.), Cognitive aspects of survey methodology: Building a bridge between disciplines. Washington: National Academy Press. Reprinted from: Not in File.Google Scholar
  40. van Oort, L., Schroder, C., & French, D. P. (2011). What do people think about when they answer the Brief Illness Perception Questionnaire? A ‘think-aloud’ study. British Journal of Health Psychology, 16(Pt 2), 231–245.CrossRefGoogle Scholar
  41. Wilkie, R., Peat, G., Thomas, E., Hooper, H., & Croft, P. R. (2005). The Keele assessment of participation: a new instrument to measure participation restriction in population studies. Combined qualitative and quantitative examination of its psychometric properties. Quality of Life Research, 14(8), 1889–1899.CrossRefGoogle Scholar
  42. Willis, G. B. (2006). Cognitive interviewing. A tool for improving questionnaire design. Quality of Life Research.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht and The International Society for Quality-of-Life Studies (ISQOLS) 2013

Authors and Affiliations

  1. 1.School of Social and Community Medicine & School of Clinical SciencesUniversity of BristolBristolUK
  2. 2.School of Oral and Dental SciencesUniversity of BristolBristolUK
  3. 3.Department of Health EconomicsUniversity of BirminghamBirminghamUK

Personalised recommendations