Quality of Life Research

, 18:1263 | Cite as

Qualitative research and content validity: developing best practices based on science and experience

  • Meryl BrodEmail author
  • Laura E. Tesler
  • Torsten L. Christensen



Establishing content validity for both new and existing patient-reported outcome (PRO) measures is central to a scientifically sound instrument development process. Methodological and logistical issues present a challenge in regard to determining the best practices for establishing content validity.


This paper provides an overview of the current state of knowledge regarding qualitative research to establish content validity based on the scientific methodological literature and authors’ experience.


Conceptual issues and frameworks for qualitative interview research, developing the interview discussion guide, reaching saturation, analysis of data, developing a theoretical model, item generation and cognitive debriefing are presented. Suggestions are offered for dealing with logistical issues regarding facilitator qualifications, ethics approval, sample recruitment, group logistics, taping and transcribing interviews, honoraria and documenting content validity.


It is hoped this paper will stimulate further discussion regarding best practices for establishing content validity so that, as the PRO field moves forward, qualitative research can be evaluated for quality and acceptability according to scientifically established principles.


Assessing content validity Patient-reported outcomes (PRO) development Qualitative research 


  1. 1.
    Nunally, J., & Bernstein, I. (1994). Psychometric theory (3rd ed., p. 104). McGraw-Hill: New York.Google Scholar
  2. 2.
    Leidy, N., & Vernon, M. (2008). Perspectives on patient-reported outcomes. Content validity and qualitative research in a changing clinical trial environment. Pharmacoeconomics, 26(5), 363–370.PubMedCrossRefGoogle Scholar
  3. 3.
    U.S. Department of Health and Human Services. (2008). Food and drug administration. Guidance for industry. Patient-reported outcome measures: Use in medical product development to support labeling claims. Rockville, MD.
  4. 4.
    Denzin, N., & Lincoln, Y. (Eds.). (2003). Collection and interpreting qualitative materials (2nd ed.). Thousand Oaks, CA: Sage.Google Scholar
  5. 5.
    Snape, D., & Spencer, L. (2004). The foundations of qualitative research. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: A guide for social science students and researchers (pp. 1–23). London: SAGE.Google Scholar
  6. 6.
    Theobald, S., & Nhlema-Simwaka, B. (2008). The research, policy and practice interface: Reflections on using applied social research to promote equity in health in Malawi. Social Science and Medicine, 67, 760–770.PubMedCrossRefGoogle Scholar
  7. 7.
    Friedland, G. H. (2006). HIV medication adherence: The intersection of biomedical, biobehavioral, and social science research and clinical practice. Journal of Acquired Immune Deficiency Syndromes, 43(Suppl 1), 53–59.Google Scholar
  8. 8.
    Greenhalgh, T., & Taylor, R. (1997). How to read a paper: Papers that go beyond numbers (qualitative research). British Medical Journal, 315, 740–743.PubMedGoogle Scholar
  9. 9.
    Firestone, W. A., & Herriott, R. E. (1983). The formalization of qualitative research: An adaptation of “soft science” to the policy world. Evaluation Review, 7, 437–466.CrossRefGoogle Scholar
  10. 10.
    Belue, R., Taylor-Richardson, K. D., Lin, J., Rivera, A. T., & Grandison, D. (2006). African Americans and participation in clinical trials: Differences in beliefs and attitudes by gender. Contemporary Clinical Trials, 27, 498–505.PubMedCrossRefGoogle Scholar
  11. 11.
    Featherstone, K., & Donavan, J. L. (1998). Random allocation or allocation at random? Patients’ perspectives of participation in a randomised controlled trial. BMJ, 317, 1177–1180.PubMedGoogle Scholar
  12. 12.
    Lawton, J., Fox, A., Fox, C., & Kinmonth, A. L. (2003). Participating in the United Kingdom prospective diabetes study (UKPDS): A qualitative study of patients’ experiences. British Journal of General Practice, 53, 394–398.PubMedGoogle Scholar
  13. 13.
    Madsen, S. M., Holm, S., & Riis, P. (2009). Attitudes towards clinical research among cancer trial participants and non-participants: An interview study using a grounded theory approach. Journal of Medical Ethics, 33, 234–240.CrossRefGoogle Scholar
  14. 14.
    Marsden, J., & Bradburn, J. (2004). Patient and clinician collaboration in the design of a national randomized breast cancer trial. Health Expectations, 7, 6–17.PubMedCrossRefGoogle Scholar
  15. 15.
    Paterniti, D. A., Chen, M. S., Chiechi, C., Beckett, L. A., Horan, N., Turrell, C., et al. (2005). Asian Americans and cancer clinical trials: A mixed-methods approach to understanding awareness and experience. Cancer Supplement, 104(12), 3015–3024.Google Scholar
  16. 16.
    Silberfeld, M., Rueda, S., Krahn, M., & Naglie, G. (2002). Content validity for dementia of three generic preference based health related quality of life instruments. Quality of Life Research, 11, 71–79.PubMedCrossRefGoogle Scholar
  17. 17.
    Waters, E., Maher, E., Salmon, L., Reddihough, D., & Boyd, R. (2005). Development of a condition-specific measure of quality of life for children with cerebral palsy: Empirical thematic data reported by parents and children. Child: Care, Health and Development, 31(2), 127–135.CrossRefGoogle Scholar
  18. 18.
    Wieringa, N. F., Peschar, J. L., Denig, P., de Graeff, P. A., & Vos, R. (2003). Connecting pre-marketing clinical research and medical practice. International Journal of Technology Assessment in Health Care, 19(1), 202–219.PubMedCrossRefGoogle Scholar
  19. 19.
    Glaser, B., & Strauss, A. (1967). The discovery of grounded theory. Chicago: Aldine Press.Google Scholar
  20. 20.
    Corbin, J., & Strauss, A. (1990). Grounded theory research: Procedures, canons and evaluative criteria. Qualitative Sociology, 13(1), 3–21.CrossRefGoogle Scholar
  21. 21.
    Charmaz, K. (2003). Qualitative interviewing and grounded theory analysis. In J. A. Holstein & J. F. Gubrium (Eds.), Inside interviewing: New lenses, new concerns (pp. 311–330). Thousand Oaks, CA: Sage.Google Scholar
  22. 22.
    Patton, M. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage.Google Scholar
  23. 23.
    McGhee, G., Marland, G. R., & Atkinson, J. (2007). Grounded theory research: Literature reviewing anf reflexivity. Journal of Advanced Nursing, 60(3), 334–342.PubMedGoogle Scholar
  24. 24.
    Corbin, J., & Strauss, A. (2007). Basics of qualitative research (3rd ed.). Newbury Park, CA: Sage.Google Scholar
  25. 25.
    Patrick, D. L., Burke, L. B., Powers, J. H., Scott, J. A., Rock, E. P., Dawisha, S., et al. (2007). Patient-reported outcomes to support medical product labeling claims: FDA perspective. Value Health, 10(Suppl 2), S125–S137.PubMedCrossRefGoogle Scholar
  26. 26.
    Morgan, D. (1996). Focus groups. Annual Review of Sociology, 22, 129–152.CrossRefGoogle Scholar
  27. 27.
    Stewart, D., Shamdasani, P. N., & Rook, D. W. (2006). Focus groups (2nd ed.). Thousand Oaks, CA: Sage.Google Scholar
  28. 28.
    Quine, S., & Cameron, I. (1995). The use of focus groups with the disabled elderly. Qualitative Health Research, 5(4), 454–462.CrossRefGoogle Scholar
  29. 29.
    Koppelman, N., & Bourjolly, J. (2001). Conducting focus groups with women with severe psychiatric disabilities: A methodological overview. Psychiatric Rehabilitation Journal, 25(2), 142–151.PubMedGoogle Scholar
  30. 30.
    Kitzinger, J. (1995). Qualitative research: Introducing focus groups. BMJ, 311, 299–302.PubMedGoogle Scholar
  31. 31.
    Greenbaum, T. (2000). Moderating focus groups: A practical guide for group facilitation. Thousand Oaks, CA: Sage.Google Scholar
  32. 32.
    Morgan, D. (1997). Focus groups as qualitative research (2nd ed.). Thousand Oaks, CA: Sage.Google Scholar
  33. 33.
    Hollander, J. (2004). The social contexts of focus groups. Journal of Contemporary Ethnography, 33(5), 602–637.CrossRefGoogle Scholar
  34. 34.
    Turner, R. R., Quittner, A. L., Parasuraman, B. M., Kallich, J. D., Cleeland, C. S., & Mayo/FDA Patient-Reported Outcomes Consensus Meeting Group. (2007). Patient-reported outcomes: Instrument development and selection issues. Value Health, 10(Suppl 2), S86–S93.PubMedCrossRefGoogle Scholar
  35. 35.
    Willis, G. B. (2004). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks: Sage.Google Scholar
  36. 36.
    Willis, G. B. (2004). Cognitive interviewing revisited: A useful technique, in theory? In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 23–44). New York: Wiley-IEEE.CrossRefGoogle Scholar
  37. 37.
    Beatty, P. (2004). The dynamics of cognitive interviewing. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 45–66). New York: Wiley-IEEE.CrossRefGoogle Scholar
  38. 38.
    Cutliffe, J. (2000). Methodological issues in grounded theory. Journal of Advanced Nursing, 31(6), 1476–1484.CrossRefGoogle Scholar
  39. 39.
    Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18(1), 59–82.CrossRefGoogle Scholar
  40. 40.
    Poland, B. (2003). Transcription quality. In J. A. Holstein & J. F. Gubrium (Eds.), Inside interviewing: New lenses, new concerns (pp. 267–288). Thousand Oaks, CA: Sage.Google Scholar
  41. 41.
    Bernard, H. R. (2005). Research methods in anthropology (4th ed.). Walnut Creek, CA: Rowman Altamira.Google Scholar
  42. 42.
    St John, W., & Johnson, P. (2000). The pros and cons of data analysis software for qualitative research. Journal of Nursing Scholarship, 32(4), 393–397.PubMedCrossRefGoogle Scholar
  43. 43.
    Jennings, B. (2007). Qualitative analysis: A case of software or ‘peopleware?’. Research in Nursing and Health, 30, 483–484.PubMedCrossRefGoogle Scholar
  44. 44.
    Morison, M., & Moir, J. (1998). The role of computer software in the analysis of qualitative data: Efficient clerk, research assistant or Trojan horse? Journal of Advanced Nursing, 28(1), 106–116.PubMedCrossRefGoogle Scholar
  45. 45.
    Ritchie, J., Spencer, L., & O’Connor, W. (2003). Carrying out qualitative analysis. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: A guide for social science students and researchers (pp. 219–262). London: Sage.Google Scholar
  46. 46.
    Hruschka, D., Schwartz, D., St John, D., Picone-Decaro, E., Jenkins, R., & Carey, J. (2004). Reliability in coding open-ended data: Lessons learned from HIV behavioral research. Field Methods 307–331.Google Scholar
  47. 47.
    Rothman, M. L., Beltran, P., Cappelleri, J. C., Lipscomb, J., Teschendorf, B., & Mayo/FDA Patient-Reported Outcomes Consensus Meeting Group. (2007). Patient-reported outcomes: Conceptual issues. Value Health, 10(Suppl 2), S66–S75.PubMedCrossRefGoogle Scholar
  48. 48.
    Bradburn, N. M., Sudman, S., & Wansink, B. (2004). Asking questions. New York: John Wiley and Sons.Google Scholar
  49. 49.
    Acquadro, C., Conway, C., Wolf, B., Anfray, C., Hareendran, A., Mear, I., et al. (2008). Development of a standardized classification system for the translations of patient-reported outcome (PRO) measures. Quality of Life Newsletter, 39, 5.Google Scholar
  50. 50.
    Frost, M. H., Reeve, B. B., Liepa, A. M., Stauffer, J. W., & Hays, R. D. (2007). What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value in Health, 10(2), S94–S105.PubMedCrossRefGoogle Scholar
  51. 51.
    Beatty, P. C., & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 1–25.Google Scholar
  52. 52.
    Willis, G. B. (1999). Cognitive interviewing: A “how to” guide. Resource document. National Cancer Institute. Accessed 2 May 2009.
  53. 53.
    Krueger, R. (1995). The future of focus groups. Qualitative Health Research, 5(4), 524–530.CrossRefGoogle Scholar
  54. 54.
    U.S. Department of Health and Human Services. (2008). Food and drug administration. CFR—Code of Federal Regulations Title 21: Part 812—Investigational Device Exemptions. Resource Document.
  55. 55.
    U.S. Department of Health and Human Services. (2008). Food and drug administration. CFR—Code of Federal Regulations Title 21: Part 812—Investigational New Drug Application. Resource Document.
  56. 56.
    Revicki, D. A., Gnanasakthy, A., & Weinfurt, K. (2007). Documenting the rationale and psychometric characteristics of patient reported outcomes for labeling and promotional claims: The PRO evidence dossier. Quality of Life Research, 16, 717–723.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2009

Authors and Affiliations

  • Meryl Brod
    • 1
    Email author
  • Laura E. Tesler
    • 1
  • Torsten L. Christensen
    • 2
  1. 1.The Brod GroupMill ValleyUSA
  2. 2.Novo Nordisk A/S, Global DevelopmentBagsværdDenmark

Personalised recommendations