Content validity in the PROMIS social-health domain: a qualitative analysis of focus-group data

  • Liana D. CastelEmail author
  • Kelly A. Williams
  • Hayden B. Bosworth
  • Susan V. Eisen
  • Elizabeth A. Hahn
  • Debra E. Irwin
  • Morgen A. R. Kelly
  • Jennifer Morse
  • Angela Stover
  • Darren A. DeWalt
  • Robert F. DeVellis



Our aim was to assess the content validity of the Patient-Reported Outcomes Measurement Information System (PROMIS) social health item banks by comparing a prespecified conceptual model with concepts that focus-group participants identified as important social-health-related outcomes. These data will inform the process of improving health-related quality-of-life measures.


Twenty-five patients with a range of social limitations due to chronic health conditions were recruited at two sites; four focus groups were conducted. Raters independently classified participants’ statements using a hierarchical, nested schema that included health-related outcomes, role performance, role satisfaction, family/friends, work, and leisure.


Key themes that emerged were fulfilling both family and work responsibilities and the distinction between activities done out of responsibility versus enjoyment. Although focus-group participants identified volunteerism and pet ownership as important social-health-related concepts, these were not in our original conceptual model. The concept of satisfaction was often found to overlap with the concept of performance.


Our conceptual model appears comprehensive but is being further refined to more appropriately (a) distinguish between responsibilities versus discretionary activities, and (b) situate the outcome of satisfaction as it relates to impairment in social and other domains of health.


Epidemiologic measurements  Focus groups Outcome assessment (health care) Qualitative research Quality of life 



Family and friends


Health-related outcome(s)


Health-related quality of life




National Institutes of Health


Patient-reported outcome(s)


Patient-Reported Outcomes Measurement Information System


Role participation performance


Role participation satisfaction


University of North Carolina at Chapel Hill




World Health Organization


Western Psychiatric Institute and Clinic (University of Pittsburgh)



The Patient-Reported Outcomes Measurement Information System (PROMIS) is a National Institutes of Health (NIH) Roadmap initiative to develop a computerized system measuring patient-reported outcomes in respondents with a wide range of chronic diseases and demographic characteristics. PROMIS was funded by cooperative agreements to a Statistical Coordinating Center (Evanston Northwestern Healthcare, PI: David Cella, PhD, U01AR52177) and six Primary Research Sites (Duke University, PI: Kevin Weinfurt, PhD, U01AR52186; University of North Carolina, PI: Darren DeWalt, MD, MPH, U01AR52181; University of Pittsburgh, PI: Paul A. Pilkonis, PhD, U01AR52155; Stanford University, PI: James Fries, MD, U01AR52158; Stony Brook University, PI: Arthur Stone, PhD, U01AR52170; and University of Washington, PI: Dagmar Amtmann, PhD, U01AR52171). NIH Science Officers on this project are William Riley, Ph.D., Susan Czajkowski, PhD, Lawrence Fine, MD, DrPH, Louis Quatrano, PhD, Bryce Reeve, PhD, James Witter, MD, and Susana Serrate-Sztein, PhD. This manuscript was reviewed by the PROMIS Publications Subcommittee prior to external peer review. See the Web site at for additional information on the PROMIS cooperative group. This work was also supported by AHRQ National Research Service Award Research Training Grant T32 HS000032–17.

We gratefully acknowledge the editorial assistance provided by David Cella of Evanston Northwestern Healthcare, as well as the data coding efforts of the three raters at the UNC site: Katherine Buysse, Jessica Dilday, and Ashley Hink. Significant acknowledgements at the Pittsburgh site include Emily Huisman, Catherine Maihoefer, and Nathan Dodds.


  1. 1.
    Food and Drug Administration. (2006). Guidance for industry: Patient-reported outcome measures: use in medical product development to support labeling claims (DRAFT). US Department of Health and Human Services Food and Drug Administration; Center for Drug Evaluation and Research (CDER); Center for Biologics Evaluation and Research (CBER); Center for Devices and Radiological Health (CDRH), February 2006. Available from: Accessed December 18, 2007.
  2. 2.
    Cella, D., Yount, S., et al. (2007). The patient-reported outcomes measurement information system (PROMIS): Progress of an NIH roadmap cooperative group during its first two years. Medical Care, 45(5 Suppl 1), S3–S11.PubMedCrossRefGoogle Scholar
  3. 3.
    DeWalt, D. A., Rothrock, N., et al. (2007). Evaluation of item candidates: The PROMIS qualitative item review. Medical Care, 45(5 Suppl 1), S12–S21.PubMedCrossRefGoogle Scholar
  4. 4.
    World Health Organization. (1946). Constitution of the World Health Organization. Geneva: World Health Organization.Google Scholar
  5. 5.
    World Health Organization. (2001). ICF: International Classification of Functioning, Disability and Health. Geneva: World Health Organization.Google Scholar
  6. 6.
    Reeve, B. B., Hays, R. D., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45(5 Suppl 1), S22–S31.PubMedCrossRefGoogle Scholar
  7. 7.
    Birchwood, M., Smith, J., et al. (1990). The Social functioning scale. The development and validation of a new scale of social adjustment for use in family intervention programmes with schizophrenic patients. The British Journal of Psychiatry, 157, 853–859.PubMedCrossRefGoogle Scholar
  8. 8.
    Dijkers, M. P., Whiteneck, G., et al. (2000). Measures of social outcomes in disability research. Archives of Physical Medicine and Rehabilitation, 81(12 Suppl 2), S63–S80.PubMedCrossRefGoogle Scholar
  9. 9.
    Eisen, S. V., Normand, S. L. T., et al. (1994). BASIS-32 and the Revised Behavioral Symptom Identification Scale (BASIS-R). In M. Maruish (Ed.), The use of psychological testing for treatment planning and outcome assessment (pp. 759–790). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  10. 10.
    Horowitz, L. M., Rosenberg, S. E., et al. (1988). Inventory of interpersonal problems: psychometric properties and clinical applications. Journal of Consulting and Clinical Psychology, 56(6), 885–892.PubMedCrossRefGoogle Scholar
  11. 11.
    Weissman, M. M., & Bothwell, S. (1976). Assessment of social adjustment by patient self-report. Archives of General Psychiatry, 33(9), 1111–1115.PubMedGoogle Scholar
  12. 12.
    World Health Organization. (2000). Disability Assessment Schedule II (WHO DAS II) Interviewer’s training manual. Geneva.Google Scholar
  13. 13.
    McDowell, I., & Newell, C. (1996). Measuring health: A guide to rating scales and questionnaires (2nd ed.). New York: Oxford University Press.Google Scholar
  14. 14.
    Cohen, S., & Syme, S. L. (1985). Issues in the study, application of social support. In S. Cohen & S. L. Syme (Eds.), Social support and health (pp. 3–22). Orlando, FL: Academic Press, Inc.Google Scholar
  15. 15.
    McCampbell, C., & Helmer, O. (1993). An experimental application of the Delphi method to the use of experts. Management Science, 9(3), 458–467.Google Scholar
  16. 16.
    Hahn, E., Cella, D., et al. (1991). Social well-being: The forgotten health status measure. Quality of Life Research, 14(9).Google Scholar
  17. 17.
    Curtis, E., & Redmond, R. (2007). Focus groups in nursing research. Nurse Res, 14(2), 25–37.PubMedGoogle Scholar
  18. 18.
    Krueger, R. (1994). Focus groups: A practical guide for applied research (2nd ed). Thousand Oaks, CA: Sage.Google Scholar
  19. 19.
    Morgan, D. (1988). Focus groups as qualitative research. Qualitative research methods (Vol 16). Newbury Park: Sage.Google Scholar
  20. 20.
    Morgan, D. (Ed.). (1993). Successful focus groups: advancing the state of the art. Newbury Park: Sage.Google Scholar
  21. 21.
    Stewart, D., Shamdasani, P., et al. (2007). Focus groups: Theory and practice. Thousand Oaks: Sage.Google Scholar
  22. 22.
    Kitzinger, J. (1995). Qualitative research. Introducing focus groups. BMJ, 311(7000), 299–302.PubMedGoogle Scholar
  23. 23.
    Bender, D., & Ewbank, D. (1994). The focus group as a tool for health research: Issues in design and analysis. Health Transition Review, 4(1), 63–79.PubMedGoogle Scholar
  24. 24.
    Muhr, T. (2004). User’s manual for ATLAS.ti 5.0. GmbH, Berlin: ATLAS.ti Scientific Software Development.Google Scholar
  25. 25.
    Carey, J. W., Morgan, M., et al. (1996). Intercoder agreement in analysis of responses to open-ended interview questions: Examples from tuberculosis research. Field Methods, 8(3), 1–5.CrossRefGoogle Scholar
  26. 26.
    Gorden, R. L. (1992). Basic interviewing skills. Long Grove: Waveland Press.Google Scholar
  27. 27.
    Kupper, L. L., & Hafner, K. B. (1989). On assessing interrater agreement for multiple attribute responses. Biometrics, 45(3), 957–967.PubMedCrossRefGoogle Scholar
  28. 28.
    Herrald, M. M., Herrald, M. M., et al. (2002). Pet ownership predicts adherence to cardiovascular rehabilitation. Journal of Applied Social Psychology, 32(6), 1107–1123.CrossRefGoogle Scholar
  29. 29.
    Frost, M. H., Reeve, B. B., et al. (2007). What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value Health, 10 (Suppl 2), S94–S105.PubMedCrossRefGoogle Scholar
  30. 30.
    Ustun, T., Chatterji, S., et al. (Eds.). (2001). Disability and culture: Universalism and diversity. Gottingen: World Health Organization.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2008

Authors and Affiliations

  • Liana D. Castel
    • 1
    Email author
  • Kelly A. Williams
    • 2
  • Hayden B. Bosworth
    • 3
  • Susan V. Eisen
    • 4
  • Elizabeth A. Hahn
    • 5
    • 6
    • 7
  • Debra E. Irwin
    • 8
  • Morgen A. R. Kelly
    • 9
  • Jennifer Morse
    • 9
  • Angela Stover
    • 9
  • Darren A. DeWalt
    • 1
    • 10
  • Robert F. DeVellis
    • 11
  1. 1.Cecil G. Sheps Center for Health Services ResearchUniversity of North Carolina at Chapel HillChapel HillUSA
  2. 2.School of Social Work Ph.D. ProgramUniversity of North Carolina at Chapel HillChapel HillUSA
  3. 3.Duke University Medical CenterDurhamUSA
  4. 4.Department of Health Policy & ManagementBoston University School of Public Health and Center for Health Quality, Outcomes & Economic Research, ENRM Veterans HospitalBedfordUSA
  5. 5.Center on Outcomes, Research and Education (CORE), Evanston Northwestern HealthcareEvanstonUSA
  6. 6.Department of Preventive Medicine, Feinberg School of MedicineNorthwestern UniversityChicagoUSA
  7. 7.Institute for Healthcare Studies, Feinberg School of MedicineNorthwestern UniversityChicagoUSA
  8. 8.Department of EpidemiologyUniversity of North Carolina at Chapel HillChapel HillUSA
  9. 9.Western Psychiatric Institute & ClinicUniversity of Pittsburgh Medical CenterPittsburghUSA
  10. 10.Division of General Medicine and Clinical EpidemiologyUniversity of North Carolina at Chapel HillChapel HillUSA
  11. 11.Department of Health Behavior and Health Education, School of Public HealthUniversity of North Carolina at Chapel Hill, Thurston Arthritis Research CenterChapel HillUSA

Personalised recommendations