Quality of Life Research

, Volume 21, Issue 5, pp 739–746 | Cite as

Content validity of patient-reported outcome measures: perspectives from a PROMIS meeting

  • Susan Magasi
  • Gery Ryan
  • Dennis Revicki
  • William Lenderking
  • Ron D. Hays
  • Meryl Brod
  • Claire Snyder
  • Maarten Boers
  • David Cella


Content validity of patient-reported outcome measures (PROs) has been a focus of debate since the 2006 publication of the U.S. FDA Draft Guidance for Industry in Patient Reported Outcome Measurement. Under the auspices of the Patient Reported Outcomes Measurement Information System (PROMIS) initiative, a working meeting on content validity was convened with leading PRO measurement experts. Platform presentations and participant discussion highlighted key issues in the content validity debate, including inconsistency in the definition and evaluation of content validity, the need for empirical research to support methodological approaches to the evaluation of content validity, and concerns that continual re-evaluation of content validity slows the pace of science and leads to the proliferation of study-specific PROs. We advocate an approach to the evaluation of content validity, which includes meticulously documented qualitative and advanced quantitative methods. To advance the science of content validity in PROs, we recommend (1) development of a consensus definition of content validity; (2) development of content validity guidelines that delineate the role of qualitative and quantitative methods and the integration of multiple perspectives; (3) empirical evaluation of generalizability of content validity across applications; and (4) use of generic measures as the foundation for PROs assessment.


PRO development Content validity Qualitative research Quantitative research 



Computer adaptive testing


Confirmatory factor analysis


Differential item functioning


United States Food and Drug Administration


Item response theory


International Society for Quality of Life Research


International Society for Pharmacoeconomics and Outcomes Research


National Institutes of Health


Patient-reported outcome


Patient Reported Outcomes Measurement Information System


Structural equation modeling



This work was funded by the National Institutes of Health through the NIH Roadmap for Medical Research, Grant (1U01-AR052177). This manuscript was reviewed by PROMIS reviewers, Paul Pilkonis, Arthur Stone, and Paul Crane, before submission for external peer review. Laurie Burke also provided thoughtful comments on an early draft.


  1. 1.
    AERA, APA, NCME. (1999). American Psychological Association. Washington, DC: Standards for educational and psychological testing.Google Scholar
  2. 2.
    U.S. Department of Health and Human Services. (2006). Food and Drug Administration draft guidance for industry on patient-reported outcome measures: use in medical product development to support labeling claims. Federal Register.Google Scholar
  3. 3.
    U.S. Department of Health and Human Services. (2009). Food and Drug Administration guidance for industry on patient-reported outcome measures: use in medical product development to support labeling claims. Federal Register.Google Scholar
  4. 4.
    Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook, K., Reeve, B., et al. (2007). The patient-reported outcomes measurement information system (PROMIS): Progress of an NIH roadmap cooperative group during the first two years. Medical Care, 45(Suppl 1), S3–S11.PubMedCrossRefGoogle Scholar
  5. 5.
    Brod, M., Tesler, L. E., & Christensen, T. L. (2009). Qualitative research and content validity: Developing best practices based on science and experience. Quality of Life Research, 18, 1263–1278.PubMedCrossRefGoogle Scholar
  6. 6.
    Lasch, K., Marquis, P., Vigneux, M., et al. (2010). PRO development: rigorous qualitative research as the crucial foundation. Quality of Life Research, 19, 1087–1096.PubMedCrossRefGoogle Scholar
  7. 7.
    Charmaz, K. (2006). Constructing grounded theory: A practical guide though qualitative analysis. Washington, DC: Sage.Google Scholar
  8. 8.
    Charmaz, K. (2003). Grounded theory: Objectivist and constructivist methods. In: G. Lincoln & M. Day (Eds.), Strategies for qualitative inquiry (2nd ed.). Thousand Oaks, CA: Sage PublicationsGoogle Scholar
  9. 9.
    Strauss, A., & Corbin, J. M. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory (2nd ed.). Thousand Oaks, CA: Sage Publications.Google Scholar
  10. 10.
    Morse, J., Barnett, N., Mayan, M., Olson, K., Spiers, J. (2002). Verification strategies for establishing reliability and validity in qualitative research. International Journal of Qualitative Methods, 1, Article 2. URL: http//www, Scholar
  11. 11.
    Bowen, G. (2008). Naturalistic inquiry and the saturation concept: A research note. Qualitative Research, 8, 137–152.CrossRefGoogle Scholar
  12. 12.
    Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? Field Methods, 18, 59–82.CrossRefGoogle Scholar
  13. 13.
    Rothman, M., Burke, L., Erickson, P., Leidy, N. K., Patrick, D. L., & Petrie, C. D. (2009). Use of existing patient-reported outcome (PRO) instruments and their modification: The ISPOR good research practices for evaluating and documenting content validity for the use of existing instruments and their modification PROTask force report. Value in Health, 12, 1075–1083.PubMedCrossRefGoogle Scholar
  14. 14.
    Triandis, H. (1994). Culture and social behavior. New York: McGraw-Hill, Inc.Google Scholar
  15. 15.
    Hays, R. D., & Fayers, P. (2005). Evaluating multi-item scales. In P. Fayers & R. D. Hays (Eds.), Assessing quality of life in clinical trials: Methods and practice (2nd ed., pp. 41–53). New York: Oxford University Press.Google Scholar
  16. 16.
    Revicki, D. A., Sorensen, S., & Wu, A. W. (1998). Reliability and validity of physical and mental health summary scores from the medical outcomes study HIV health survey. Medical Care, 36, 126–137.PubMedCrossRefGoogle Scholar
  17. 17.
    Joeskog, K. G. (2005). Principles and practice of structural equation modeling (2nd ed.). New York: Guilford Press.Google Scholar
  18. 18.
    Hays, R. D., Revicki, D. A., & Coyne, K. S. (2005). Application of structural equation modeling to health outcomes research. Evaluation and the Health Professions, 28, 295–309.CrossRefGoogle Scholar
  19. 19.
    Kline, R. (2005). Principles and practice of structural equation modeling (2nd ed.). New York: Guilford Press.Google Scholar
  20. 20.
    Stull, D. E. (2008). Analyzing growth and change: Latent variable growth curve modeling with an application to clinical trials. Quality of Life Research, 17, 47–59.PubMedCrossRefGoogle Scholar
  21. 21.
    Stull, D., Vernon, M. K., Legg, J. C., Viswanathan, H. N., Fairclough, D., & Revicki, D. A. (2010). Use of linear growth curve models for assessing the effects of darbepoetin alpha on hemoglobin and fatigue. Contemporary Clinical Trials, 31, 172–179.PubMedCrossRefGoogle Scholar
  22. 22.
    Embretson, S., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  23. 23.
    Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the patient-reported outcome measurement information system (PROMIS). Medical Care, 45(Suppl 1), S22–S31.PubMedCrossRefGoogle Scholar
  24. 24.
    DeWalt, D., Rothrock, N. P., Yount, S. P., & Stone, A. A. P. (2007). on behalf of the PCG. Evaluation of item candidates: The PROMIS qualitative item review. Medical Care, 45(Suppl 1), S12–S21.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  • Susan Magasi
    • 1
  • Gery Ryan
    • 2
  • Dennis Revicki
    • 3
  • William Lenderking
    • 3
  • Ron D. Hays
    • 4
  • Meryl Brod
    • 5
  • Claire Snyder
    • 6
  • Maarten Boers
    • 7
  • David Cella
    • 1
  1. 1.Department of Medical Social SciencesFeinberg School of Medicine Northwestern UniversityChicagoUSA
  2. 2.Rand CorporationSanta MonicaUSA
  3. 3.United BioSource CorporationBethesdaUSA
  4. 4.Department of MedicineUniversity of California, Los AngelesLos AngelesUSA
  5. 5.The Brod GroupMill ValleyUSA
  6. 6.Division of General Internal MedicineJohns Hopkins School of MedicineBaltimoreUSA
  7. 7.Department of Epidemiology and BiostatisticsVrije Universiteit (VU) University Medical CenterAmsterdamThe Netherlands

Personalised recommendations