Qualitative research and content validity: developing best practices based on science and experience
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.Get Access
Establishing content validity for both new and existing patient-reported outcome (PRO) measures is central to a scientifically sound instrument development process. Methodological and logistical issues present a challenge in regard to determining the best practices for establishing content validity.
This paper provides an overview of the current state of knowledge regarding qualitative research to establish content validity based on the scientific methodological literature and authors’ experience.
Conceptual issues and frameworks for qualitative interview research, developing the interview discussion guide, reaching saturation, analysis of data, developing a theoretical model, item generation and cognitive debriefing are presented. Suggestions are offered for dealing with logistical issues regarding facilitator qualifications, ethics approval, sample recruitment, group logistics, taping and transcribing interviews, honoraria and documenting content validity.
It is hoped this paper will stimulate further discussion regarding best practices for establishing content validity so that, as the PRO field moves forward, qualitative research can be evaluated for quality and acceptability according to scientifically established principles.
- Nunally, J., & Bernstein, I. (1994). Psychometric theory (3rd ed., p. 104). McGraw-Hill: New York.
- Leidy, N., & Vernon, M. (2008). Perspectives on patient-reported outcomes. Content validity and qualitative research in a changing clinical trial environment. Pharmacoeconomics, 26(5), 363–370. CrossRef
- U.S. Department of Health and Human Services. (2008). Food and drug administration. Guidance for industry. Patient-reported outcome measures: Use in medical product development to support labeling claims. Rockville, MD. http://www.fda.gov/cder/guidance/index.htm.
- Denzin, N., & Lincoln, Y. (Eds.). (2003). Collection and interpreting qualitative materials (2nd ed.). Thousand Oaks, CA: Sage.
- Snape, D., & Spencer, L. (2004). The foundations of qualitative research. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: A guide for social science students and researchers (pp. 1–23). London: SAGE.
- Theobald, S., & Nhlema-Simwaka, B. (2008). The research, policy and practice interface: Reflections on using applied social research to promote equity in health in Malawi. Social Science and Medicine, 67, 760–770. CrossRef
- Friedland, G. H. (2006). HIV medication adherence: The intersection of biomedical, biobehavioral, and social science research and clinical practice. Journal of Acquired Immune Deficiency Syndromes, 43(Suppl 1), 53–59.
- Greenhalgh, T., & Taylor, R. (1997). How to read a paper: Papers that go beyond numbers (qualitative research). British Medical Journal, 315, 740–743.
- Firestone, W. A., & Herriott, R. E. (1983). The formalization of qualitative research: An adaptation of “soft science” to the policy world. Evaluation Review, 7, 437–466. CrossRef
- Belue, R., Taylor-Richardson, K. D., Lin, J., Rivera, A. T., & Grandison, D. (2006). African Americans and participation in clinical trials: Differences in beliefs and attitudes by gender. Contemporary Clinical Trials, 27, 498–505. CrossRef
- Featherstone, K., & Donavan, J. L. (1998). Random allocation or allocation at random? Patients’ perspectives of participation in a randomised controlled trial. BMJ, 317, 1177–1180.
- Lawton, J., Fox, A., Fox, C., & Kinmonth, A. L. (2003). Participating in the United Kingdom prospective diabetes study (UKPDS): A qualitative study of patients’ experiences. British Journal of General Practice, 53, 394–398.
- Madsen, S. M., Holm, S., & Riis, P. (2009). Attitudes towards clinical research among cancer trial participants and non-participants: An interview study using a grounded theory approach. Journal of Medical Ethics, 33, 234–240. CrossRef
- Marsden, J., & Bradburn, J. (2004). Patient and clinician collaboration in the design of a national randomized breast cancer trial. Health Expectations, 7, 6–17. CrossRef
- Paterniti, D. A., Chen, M. S., Chiechi, C., Beckett, L. A., Horan, N., Turrell, C., et al. (2005). Asian Americans and cancer clinical trials: A mixed-methods approach to understanding awareness and experience. Cancer Supplement, 104(12), 3015–3024.
- Silberfeld, M., Rueda, S., Krahn, M., & Naglie, G. (2002). Content validity for dementia of three generic preference based health related quality of life instruments. Quality of Life Research, 11, 71–79. CrossRef
- Waters, E., Maher, E., Salmon, L., Reddihough, D., & Boyd, R. (2005). Development of a condition-specific measure of quality of life for children with cerebral palsy: Empirical thematic data reported by parents and children. Child: Care, Health and Development, 31(2), 127–135. CrossRef
- Wieringa, N. F., Peschar, J. L., Denig, P., de Graeff, P. A., & Vos, R. (2003). Connecting pre-marketing clinical research and medical practice. International Journal of Technology Assessment in Health Care, 19(1), 202–219. CrossRef
- Glaser, B., & Strauss, A. (1967). The discovery of grounded theory. Chicago: Aldine Press.
- Corbin, J., & Strauss, A. (1990). Grounded theory research: Procedures, canons and evaluative criteria. Qualitative Sociology, 13(1), 3–21. CrossRef
- Charmaz, K. (2003). Qualitative interviewing and grounded theory analysis. In J. A. Holstein & J. F. Gubrium (Eds.), Inside interviewing: New lenses, new concerns (pp. 311–330). Thousand Oaks, CA: Sage.
- Patton, M. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage.
- McGhee, G., Marland, G. R., & Atkinson, J. (2007). Grounded theory research: Literature reviewing anf reflexivity. Journal of Advanced Nursing, 60(3), 334–342.
- Corbin, J., & Strauss, A. (2007). Basics of qualitative research (3rd ed.). Newbury Park, CA: Sage.
- Patrick, D. L., Burke, L. B., Powers, J. H., Scott, J. A., Rock, E. P., Dawisha, S., et al. (2007). Patient-reported outcomes to support medical product labeling claims: FDA perspective. Value Health, 10(Suppl 2), S125–S137. CrossRef
- Morgan, D. (1996). Focus groups. Annual Review of Sociology, 22, 129–152. CrossRef
- Stewart, D., Shamdasani, P. N., & Rook, D. W. (2006). Focus groups (2nd ed.). Thousand Oaks, CA: Sage.
- Quine, S., & Cameron, I. (1995). The use of focus groups with the disabled elderly. Qualitative Health Research, 5(4), 454–462. CrossRef
- Koppelman, N., & Bourjolly, J. (2001). Conducting focus groups with women with severe psychiatric disabilities: A methodological overview. Psychiatric Rehabilitation Journal, 25(2), 142–151.
- Kitzinger, J. (1995). Qualitative research: Introducing focus groups. BMJ, 311, 299–302.
- Greenbaum, T. (2000). Moderating focus groups: A practical guide for group facilitation. Thousand Oaks, CA: Sage.
- Morgan, D. (1997). Focus groups as qualitative research (2nd ed.). Thousand Oaks, CA: Sage.
- Hollander, J. (2004). The social contexts of focus groups. Journal of Contemporary Ethnography, 33(5), 602–637. CrossRef
- Turner, R. R., Quittner, A. L., Parasuraman, B. M., Kallich, J. D., Cleeland, C. S., & Mayo/FDA Patient-Reported Outcomes Consensus Meeting Group. (2007). Patient-reported outcomes: Instrument development and selection issues. Value Health, 10(Suppl 2), S86–S93. CrossRef
- Willis, G. B. (2004). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks: Sage.
- Willis, G. B. (2004). Cognitive interviewing revisited: A useful technique, in theory? In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 23–44). New York: Wiley-IEEE. CrossRef
- Beatty, P. (2004). The dynamics of cognitive interviewing. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 45–66). New York: Wiley-IEEE. CrossRef
- Cutliffe, J. (2000). Methodological issues in grounded theory. Journal of Advanced Nursing, 31(6), 1476–1484. CrossRef
- Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18(1), 59–82. CrossRef
- Poland, B. (2003). Transcription quality. In J. A. Holstein & J. F. Gubrium (Eds.), Inside interviewing: New lenses, new concerns (pp. 267–288). Thousand Oaks, CA: Sage.
- Bernard, H. R. (2005). Research methods in anthropology (4th ed.). Walnut Creek, CA: Rowman Altamira.
- St John, W., & Johnson, P. (2000). The pros and cons of data analysis software for qualitative research. Journal of Nursing Scholarship, 32(4), 393–397. CrossRef
- Jennings, B. (2007). Qualitative analysis: A case of software or ‘peopleware?’. Research in Nursing and Health, 30, 483–484. CrossRef
- Morison, M., & Moir, J. (1998). The role of computer software in the analysis of qualitative data: Efficient clerk, research assistant or Trojan horse? Journal of Advanced Nursing, 28(1), 106–116. CrossRef
- Ritchie, J., Spencer, L., & O’Connor, W. (2003). Carrying out qualitative analysis. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: A guide for social science students and researchers (pp. 219–262). London: Sage.
- Hruschka, D., Schwartz, D., St John, D., Picone-Decaro, E., Jenkins, R., & Carey, J. (2004). Reliability in coding open-ended data: Lessons learned from HIV behavioral research. Field Methods 307–331.
- Rothman, M. L., Beltran, P., Cappelleri, J. C., Lipscomb, J., Teschendorf, B., & Mayo/FDA Patient-Reported Outcomes Consensus Meeting Group. (2007). Patient-reported outcomes: Conceptual issues. Value Health, 10(Suppl 2), S66–S75. CrossRef
- Bradburn, N. M., Sudman, S., & Wansink, B. (2004). Asking questions. New York: John Wiley and Sons.
- Acquadro, C., Conway, C., Wolf, B., Anfray, C., Hareendran, A., Mear, I., et al. (2008). Development of a standardized classification system for the translations of patient-reported outcome (PRO) measures. Quality of Life Newsletter, 39, 5.
- Frost, M. H., Reeve, B. B., Liepa, A. M., Stauffer, J. W., & Hays, R. D. (2007). What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value in Health, 10(2), S94–S105. CrossRef
- Beatty, P. C., & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 1–25.
- Willis, G. B. (1999). Cognitive interviewing: A “how to” guide. Resource document. National Cancer Institute. http://appliedresearch.cancer.gov/areas/cognitive/interview.pdf. Accessed 2 May 2009.
- Krueger, R. (1995). The future of focus groups. Qualitative Health Research, 5(4), 524–530. CrossRef
- U.S. Department of Health and Human Services. (2008). Food and drug administration. CFR—Code of Federal Regulations Title 21: Part 812—Investigational Device Exemptions. Resource Document. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfCFR/CFRSearch.cfm?fr=812.140.
- U.S. Department of Health and Human Services. (2008). Food and drug administration. CFR—Code of Federal Regulations Title 21: Part 812—Investigational New Drug Application. Resource Document. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfCFR/CFRSearch.cfm?fr=312.62.
- Revicki, D. A., Gnanasakthy, A., & Weinfurt, K. (2007). Documenting the rationale and psychometric characteristics of patient reported outcomes for labeling and promotional claims: The PRO evidence dossier. Quality of Life Research, 16, 717–723. CrossRef
- Qualitative research and content validity: developing best practices based on science and experience
Quality of Life Research
Volume 18, Issue 9 , pp 1263-1278
- Cover Date
- Print ISSN
- Online ISSN
- Springer Netherlands
- Additional Links
- Assessing content validity
- Patient-reported outcomes (PRO) development
- Qualitative research
- Industry Sectors