Skip to main content

Advertisement

SpringerLink
Log in

Qualitative research and content validity: developing best practices based on science and experience

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

Establishing content validity for both new and existing patient-reported outcome (PRO) measures is central to a scientifically sound instrument development process. Methodological and logistical issues present a challenge in regard to determining the best practices for establishing content validity.

Methods

This paper provides an overview of the current state of knowledge regarding qualitative research to establish content validity based on the scientific methodological literature and authors’ experience.

Results

Conceptual issues and frameworks for qualitative interview research, developing the interview discussion guide, reaching saturation, analysis of data, developing a theoretical model, item generation and cognitive debriefing are presented. Suggestions are offered for dealing with logistical issues regarding facilitator qualifications, ethics approval, sample recruitment, group logistics, taping and transcribing interviews, honoraria and documenting content validity.

Conclusions

It is hoped this paper will stimulate further discussion regarding best practices for establishing content validity so that, as the PRO field moves forward, qualitative research can be evaluated for quality and acceptability according to scientifically established principles.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Nunally, J., & Bernstein, I. (1994). Psychometric theory (3rd ed., p. 104). McGraw-Hill: New York.

    Google Scholar 

  2. Leidy, N., & Vernon, M. (2008). Perspectives on patient-reported outcomes. Content validity and qualitative research in a changing clinical trial environment. Pharmacoeconomics, 26(5), 363–370.

    Article  PubMed  Google Scholar 

  3. U.S. Department of Health and Human Services. (2008). Food and drug administration. Guidance for industry. Patient-reported outcome measures: Use in medical product development to support labeling claims. Rockville, MD. http://www.fda.gov/cder/guidance/index.htm.

  4. Denzin, N., & Lincoln, Y. (Eds.). (2003). Collection and interpreting qualitative materials (2nd ed.). Thousand Oaks, CA: Sage.

    Google Scholar 

  5. Snape, D., & Spencer, L. (2004). The foundations of qualitative research. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: A guide for social science students and researchers (pp. 1–23). London: SAGE.

    Google Scholar 

  6. Theobald, S., & Nhlema-Simwaka, B. (2008). The research, policy and practice interface: Reflections on using applied social research to promote equity in health in Malawi. Social Science and Medicine, 67, 760–770.

    Article  PubMed  Google Scholar 

  7. Friedland, G. H. (2006). HIV medication adherence: The intersection of biomedical, biobehavioral, and social science research and clinical practice. Journal of Acquired Immune Deficiency Syndromes, 43(Suppl 1), 53–59.

    Google Scholar 

  8. Greenhalgh, T., & Taylor, R. (1997). How to read a paper: Papers that go beyond numbers (qualitative research). British Medical Journal, 315, 740–743.

    PubMed  CAS  Google Scholar 

  9. Firestone, W. A., & Herriott, R. E. (1983). The formalization of qualitative research: An adaptation of “soft science” to the policy world. Evaluation Review, 7, 437–466.

    Article  Google Scholar 

  10. Belue, R., Taylor-Richardson, K. D., Lin, J., Rivera, A. T., & Grandison, D. (2006). African Americans and participation in clinical trials: Differences in beliefs and attitudes by gender. Contemporary Clinical Trials, 27, 498–505.

    Article  PubMed  CAS  Google Scholar 

  11. Featherstone, K., & Donavan, J. L. (1998). Random allocation or allocation at random? Patients’ perspectives of participation in a randomised controlled trial. BMJ, 317, 1177–1180.

    PubMed  CAS  Google Scholar 

  12. Lawton, J., Fox, A., Fox, C., & Kinmonth, A. L. (2003). Participating in the United Kingdom prospective diabetes study (UKPDS): A qualitative study of patients’ experiences. British Journal of General Practice, 53, 394–398.

    PubMed  Google Scholar 

  13. Madsen, S. M., Holm, S., & Riis, P. (2009). Attitudes towards clinical research among cancer trial participants and non-participants: An interview study using a grounded theory approach. Journal of Medical Ethics, 33, 234–240.

    Article  Google Scholar 

  14. Marsden, J., & Bradburn, J. (2004). Patient and clinician collaboration in the design of a national randomized breast cancer trial. Health Expectations, 7, 6–17.

    Article  PubMed  Google Scholar 

  15. Paterniti, D. A., Chen, M. S., Chiechi, C., Beckett, L. A., Horan, N., Turrell, C., et al. (2005). Asian Americans and cancer clinical trials: A mixed-methods approach to understanding awareness and experience. Cancer Supplement, 104(12), 3015–3024.

    Google Scholar 

  16. Silberfeld, M., Rueda, S., Krahn, M., & Naglie, G. (2002). Content validity for dementia of three generic preference based health related quality of life instruments. Quality of Life Research, 11, 71–79.

    Article  PubMed  Google Scholar 

  17. Waters, E., Maher, E., Salmon, L., Reddihough, D., & Boyd, R. (2005). Development of a condition-specific measure of quality of life for children with cerebral palsy: Empirical thematic data reported by parents and children. Child: Care, Health and Development, 31(2), 127–135.

    Article  CAS  Google Scholar 

  18. Wieringa, N. F., Peschar, J. L., Denig, P., de Graeff, P. A., & Vos, R. (2003). Connecting pre-marketing clinical research and medical practice. International Journal of Technology Assessment in Health Care, 19(1), 202–219.

    Article  PubMed  Google Scholar 

  19. Glaser, B., & Strauss, A. (1967). The discovery of grounded theory. Chicago: Aldine Press.

    Google Scholar 

  20. Corbin, J., & Strauss, A. (1990). Grounded theory research: Procedures, canons and evaluative criteria. Qualitative Sociology, 13(1), 3–21.

    Article  Google Scholar 

  21. Charmaz, K. (2003). Qualitative interviewing and grounded theory analysis. In J. A. Holstein & J. F. Gubrium (Eds.), Inside interviewing: New lenses, new concerns (pp. 311–330). Thousand Oaks, CA: Sage.

    Google Scholar 

  22. Patton, M. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage.

    Google Scholar 

  23. McGhee, G., Marland, G. R., & Atkinson, J. (2007). Grounded theory research: Literature reviewing anf reflexivity. Journal of Advanced Nursing, 60(3), 334–342.

    PubMed  Google Scholar 

  24. Corbin, J., & Strauss, A. (2007). Basics of qualitative research (3rd ed.). Newbury Park, CA: Sage.

    Google Scholar 

  25. Patrick, D. L., Burke, L. B., Powers, J. H., Scott, J. A., Rock, E. P., Dawisha, S., et al. (2007). Patient-reported outcomes to support medical product labeling claims: FDA perspective. Value Health, 10(Suppl 2), S125–S137.

    Article  PubMed  Google Scholar 

  26. Morgan, D. (1996). Focus groups. Annual Review of Sociology, 22, 129–152.

    Article  Google Scholar 

  27. Stewart, D., Shamdasani, P. N., & Rook, D. W. (2006). Focus groups (2nd ed.). Thousand Oaks, CA: Sage.

    Google Scholar 

  28. Quine, S., & Cameron, I. (1995). The use of focus groups with the disabled elderly. Qualitative Health Research, 5(4), 454–462.

    Article  Google Scholar 

  29. Koppelman, N., & Bourjolly, J. (2001). Conducting focus groups with women with severe psychiatric disabilities: A methodological overview. Psychiatric Rehabilitation Journal, 25(2), 142–151.

    PubMed  CAS  Google Scholar 

  30. Kitzinger, J. (1995). Qualitative research: Introducing focus groups. BMJ, 311, 299–302.

    PubMed  CAS  Google Scholar 

  31. Greenbaum, T. (2000). Moderating focus groups: A practical guide for group facilitation. Thousand Oaks, CA: Sage.

    Google Scholar 

  32. Morgan, D. (1997). Focus groups as qualitative research (2nd ed.). Thousand Oaks, CA: Sage.

    Google Scholar 

  33. Hollander, J. (2004). The social contexts of focus groups. Journal of Contemporary Ethnography, 33(5), 602–637.

    Article  Google Scholar 

  34. Turner, R. R., Quittner, A. L., Parasuraman, B. M., Kallich, J. D., Cleeland, C. S., & Mayo/FDA Patient-Reported Outcomes Consensus Meeting Group. (2007). Patient-reported outcomes: Instrument development and selection issues. Value Health, 10(Suppl 2), S86–S93.

    Article  PubMed  Google Scholar 

  35. Willis, G. B. (2004). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks: Sage.

    Google Scholar 

  36. Willis, G. B. (2004). Cognitive interviewing revisited: A useful technique, in theory? In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 23–44). New York: Wiley-IEEE.

    Chapter  Google Scholar 

  37. Beatty, P. (2004). The dynamics of cognitive interviewing. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 45–66). New York: Wiley-IEEE.

    Chapter  Google Scholar 

  38. Cutliffe, J. (2000). Methodological issues in grounded theory. Journal of Advanced Nursing, 31(6), 1476–1484.

    Article  Google Scholar 

  39. Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18(1), 59–82.

    Article  Google Scholar 

  40. Poland, B. (2003). Transcription quality. In J. A. Holstein & J. F. Gubrium (Eds.), Inside interviewing: New lenses, new concerns (pp. 267–288). Thousand Oaks, CA: Sage.

    Google Scholar 

  41. Bernard, H. R. (2005). Research methods in anthropology (4th ed.). Walnut Creek, CA: Rowman Altamira.

    Google Scholar 

  42. St John, W., & Johnson, P. (2000). The pros and cons of data analysis software for qualitative research. Journal of Nursing Scholarship, 32(4), 393–397.

    Article  PubMed  CAS  Google Scholar 

  43. Jennings, B. (2007). Qualitative analysis: A case of software or ‘peopleware?’. Research in Nursing and Health, 30, 483–484.

    Article  PubMed  Google Scholar 

  44. Morison, M., & Moir, J. (1998). The role of computer software in the analysis of qualitative data: Efficient clerk, research assistant or Trojan horse? Journal of Advanced Nursing, 28(1), 106–116.

    Article  PubMed  CAS  Google Scholar 

  45. Ritchie, J., Spencer, L., & O’Connor, W. (2003). Carrying out qualitative analysis. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: A guide for social science students and researchers (pp. 219–262). London: Sage.

    Google Scholar 

  46. Hruschka, D., Schwartz, D., St John, D., Picone-Decaro, E., Jenkins, R., & Carey, J. (2004). Reliability in coding open-ended data: Lessons learned from HIV behavioral research. Field Methods 307–331.

  47. Rothman, M. L., Beltran, P., Cappelleri, J. C., Lipscomb, J., Teschendorf, B., & Mayo/FDA Patient-Reported Outcomes Consensus Meeting Group. (2007). Patient-reported outcomes: Conceptual issues. Value Health, 10(Suppl 2), S66–S75.

    Article  PubMed  Google Scholar 

  48. Bradburn, N. M., Sudman, S., & Wansink, B. (2004). Asking questions. New York: John Wiley and Sons.

    Google Scholar 

  49. Acquadro, C., Conway, C., Wolf, B., Anfray, C., Hareendran, A., Mear, I., et al. (2008). Development of a standardized classification system for the translations of patient-reported outcome (PRO) measures. Quality of Life Newsletter, 39, 5.

    Google Scholar 

  50. Frost, M. H., Reeve, B. B., Liepa, A. M., Stauffer, J. W., & Hays, R. D. (2007). What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value in Health, 10(2), S94–S105.

    Article  PubMed  Google Scholar 

  51. Beatty, P. C., & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 1–25.

  52. Willis, G. B. (1999). Cognitive interviewing: A “how to” guide. Resource document. National Cancer Institute. http://appliedresearch.cancer.gov/areas/cognitive/interview.pdf. Accessed 2 May 2009.

  53. Krueger, R. (1995). The future of focus groups. Qualitative Health Research, 5(4), 524–530.

    Article  Google Scholar 

  54. U.S. Department of Health and Human Services. (2008). Food and drug administration. CFR—Code of Federal Regulations Title 21: Part 812—Investigational Device Exemptions. Resource Document. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfCFR/CFRSearch.cfm?fr=812.140.

  55. U.S. Department of Health and Human Services. (2008). Food and drug administration. CFR—Code of Federal Regulations Title 21: Part 812—Investigational New Drug Application. Resource Document. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfCFR/CFRSearch.cfm?fr=312.62.

  56. Revicki, D. A., Gnanasakthy, A., & Weinfurt, K. (2007). Documenting the rationale and psychometric characteristics of patient reported outcomes for labeling and promotional claims: The PRO evidence dossier. Quality of Life Research, 16, 717–723.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The Brod Group, 219 Julia Avenue, Mill Valley, CA, 94941, USA

    Meryl Brod & Laura E. Tesler

  2. Novo Nordisk A/S, Global Development, Krogshøjvej 29, 2880, Bagsværd, Denmark

    Torsten L. Christensen

Authors
  1. Meryl Brod
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laura E. Tesler
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Torsten L. Christensen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meryl Brod.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Brod, M., Tesler, L.E. & Christensen, T.L. Qualitative research and content validity: developing best practices based on science and experience. Qual Life Res 18, 1263–1278 (2009). https://doi.org/10.1007/s11136-009-9540-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-009-9540-9

Keywords

Search

Navigation