Skip to main content

Advertisement

Log in

Content validity of patient-reported outcome measures: perspectives from a PROMIS meeting

  • Review
  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Content validity of patient-reported outcome measures (PROs) has been a focus of debate since the 2006 publication of the U.S. FDA Draft Guidance for Industry in Patient Reported Outcome Measurement. Under the auspices of the Patient Reported Outcomes Measurement Information System (PROMIS) initiative, a working meeting on content validity was convened with leading PRO measurement experts. Platform presentations and participant discussion highlighted key issues in the content validity debate, including inconsistency in the definition and evaluation of content validity, the need for empirical research to support methodological approaches to the evaluation of content validity, and concerns that continual re-evaluation of content validity slows the pace of science and leads to the proliferation of study-specific PROs. We advocate an approach to the evaluation of content validity, which includes meticulously documented qualitative and advanced quantitative methods. To advance the science of content validity in PROs, we recommend (1) development of a consensus definition of content validity; (2) development of content validity guidelines that delineate the role of qualitative and quantitative methods and the integration of multiple perspectives; (3) empirical evaluation of generalizability of content validity across applications; and (4) use of generic measures as the foundation for PROs assessment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Abbreviations

CAT:

Computer adaptive testing

CFA:

Confirmatory factor analysis

DIF:

Differential item functioning

FDA:

United States Food and Drug Administration

IRT:

Item response theory

ISOQOL:

International Society for Quality of Life Research

ISPOR:

International Society for Pharmacoeconomics and Outcomes Research

NIH:

National Institutes of Health

PRO:

Patient-reported outcome

PROMIS:

Patient Reported Outcomes Measurement Information System

SEM:

Structural equation modeling

References

  1. AERA, APA, NCME. (1999). American Psychological Association. Washington, DC: Standards for educational and psychological testing.

    Google Scholar 

  2. U.S. Department of Health and Human Services. (2006). Food and Drug Administration draft guidance for industry on patient-reported outcome measures: use in medical product development to support labeling claims. Federal Register.

  3. U.S. Department of Health and Human Services. (2009). Food and Drug Administration guidance for industry on patient-reported outcome measures: use in medical product development to support labeling claims. Federal Register.

  4. Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook, K., Reeve, B., et al. (2007). The patient-reported outcomes measurement information system (PROMIS): Progress of an NIH roadmap cooperative group during the first two years. Medical Care, 45(Suppl 1), S3–S11.

    Article  PubMed  Google Scholar 

  5. Brod, M., Tesler, L. E., & Christensen, T. L. (2009). Qualitative research and content validity: Developing best practices based on science and experience. Quality of Life Research, 18, 1263–1278.

    Article  PubMed  Google Scholar 

  6. Lasch, K., Marquis, P., Vigneux, M., et al. (2010). PRO development: rigorous qualitative research as the crucial foundation. Quality of Life Research, 19, 1087–1096.

    Article  PubMed  Google Scholar 

  7. Charmaz, K. (2006). Constructing grounded theory: A practical guide though qualitative analysis. Washington, DC: Sage.

    Google Scholar 

  8. Charmaz, K. (2003). Grounded theory: Objectivist and constructivist methods. In: G. Lincoln & M. Day (Eds.), Strategies for qualitative inquiry (2nd ed.). Thousand Oaks, CA: Sage Publications

  9. Strauss, A., & Corbin, J. M. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory (2nd ed.). Thousand Oaks, CA: Sage Publications.

    Google Scholar 

  10. Morse, J., Barnett, N., Mayan, M., Olson, K., Spiers, J. (2002). Verification strategies for establishing reliability and validity in qualitative research. International Journal of Qualitative Methods, 1, Article 2. URL: http//www,ualberta.ca/~ijqm.

  11. Bowen, G. (2008). Naturalistic inquiry and the saturation concept: A research note. Qualitative Research, 8, 137–152.

    Article  Google Scholar 

  12. Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? Field Methods, 18, 59–82.

    Article  Google Scholar 

  13. Rothman, M., Burke, L., Erickson, P., Leidy, N. K., Patrick, D. L., & Petrie, C. D. (2009). Use of existing patient-reported outcome (PRO) instruments and their modification: The ISPOR good research practices for evaluating and documenting content validity for the use of existing instruments and their modification PROTask force report. Value in Health, 12, 1075–1083.

    Article  PubMed  Google Scholar 

  14. Triandis, H. (1994). Culture and social behavior. New York: McGraw-Hill, Inc.

    Google Scholar 

  15. Hays, R. D., & Fayers, P. (2005). Evaluating multi-item scales. In P. Fayers & R. D. Hays (Eds.), Assessing quality of life in clinical trials: Methods and practice (2nd ed., pp. 41–53). New York: Oxford University Press.

    Google Scholar 

  16. Revicki, D. A., Sorensen, S., & Wu, A. W. (1998). Reliability and validity of physical and mental health summary scores from the medical outcomes study HIV health survey. Medical Care, 36, 126–137.

    Article  PubMed  CAS  Google Scholar 

  17. Joeskog, K. G. (2005). Principles and practice of structural equation modeling (2nd ed.). New York: Guilford Press.

    Google Scholar 

  18. Hays, R. D., Revicki, D. A., & Coyne, K. S. (2005). Application of structural equation modeling to health outcomes research. Evaluation and the Health Professions, 28, 295–309.

    Article  Google Scholar 

  19. Kline, R. (2005). Principles and practice of structural equation modeling (2nd ed.). New York: Guilford Press.

    Google Scholar 

  20. Stull, D. E. (2008). Analyzing growth and change: Latent variable growth curve modeling with an application to clinical trials. Quality of Life Research, 17, 47–59.

    Article  PubMed  Google Scholar 

  21. Stull, D., Vernon, M. K., Legg, J. C., Viswanathan, H. N., Fairclough, D., & Revicki, D. A. (2010). Use of linear growth curve models for assessing the effects of darbepoetin alpha on hemoglobin and fatigue. Contemporary Clinical Trials, 31, 172–179.

    Article  PubMed  CAS  Google Scholar 

  22. Embretson, S., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  23. Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the patient-reported outcome measurement information system (PROMIS). Medical Care, 45(Suppl 1), S22–S31.

    Article  PubMed  Google Scholar 

  24. DeWalt, D., Rothrock, N. P., Yount, S. P., & Stone, A. A. P. (2007). on behalf of the PCG. Evaluation of item candidates: The PROMIS qualitative item review. Medical Care, 45(Suppl 1), S12–S21.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This work was funded by the National Institutes of Health through the NIH Roadmap for Medical Research, Grant (1U01-AR052177). This manuscript was reviewed by PROMIS reviewers, Paul Pilkonis, Arthur Stone, and Paul Crane, before submission for external peer review. Laurie Burke also provided thoughtful comments on an early draft.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Susan Magasi.

Appendix 1: Conference participants

Appendix 1: Conference participants

Nancy Amicangelo (patient representative); Dagmar Amtmann, PhD, University of Washington; Meryl Brod, PhD, The Brod Group; Maarten Boers, MD, PhD, Vrije Universiteit; Laurie B. Burke, RPh, MPH OND/CDER/FDA; David Cella, PhD, Northwestern University; Jill Cyranowski, PhD, University of Pittsburgh Medical Center; Susan Czajkowski, PhD, National Institutes of Health; Darren DeWalt, MD, MPH. University of North Carolina; Jacqueline Dunbar-Jacob, PhD, RN, FAAN, University of Pittsburg; Sofia Garcia, PhD, Northwestern University; Richard Gershon, PhD, Northwestern University; Ari Gnanasakthy, PhD, Novartis Pharmaceuticals Corporation; Ron D. Hays, PhD, University of California at Los Angeles; Thomas Hilton, PhD, National Institute on Drug Abuse; Laura Lee Johnson, PhD, National Center for Complementary and Alternative Medicine; Nancy Kline Leidy, PhD, United BioSource Corporation; Eswat Krishnan, MD, Stanford University; William Lenderking, PhD, United BioSource Corporation (ISOQOL Representative); Amye Leong, MBA (Patient Representative); Mary Lynn, PhD, University of North Carolina at Chapel Hill; Susan Magasi, PhD, Northwestern University; Richard Moxley III, MD, University of Rochester Medical Center; Bhash Parasuramen, PhD, AstraZeneca Pharmaceuticals; Donald Patrick, PhD, University of Washington; Charles D. Petrie, PhD, Pfizer Global Development Headquarters (ISOPOR representative); Theodore Pincus, MD, NYU-Hospital for Joint Disease; Louis Quatrano, PhD, National Institute of Child Health and Human Development; Kenneth Rasinski, PhD, University of Chicago; Bryce Reeve, PhD, National Cancer Institute; William Riley, PhD, National Institute of Mental Health; Dennis Revicki, PhD, United BioSource Corporation; Margaret Rothman, PhD, Johnson & Johnson Pharmaceutical Services, LLC; Nan Rothrock, PhD, Northwestern University; Gery Ryan, PhD, Rand Corporation; Jane Scott, PhD, Mapi Values (ISOQOL Representative); Claire Snyder, PhD, Johns Hopkins School of Medicine; Ruth E.K. Stein, MD, AECOM/CHAM; Arthur Stone, PhD, Stony Brook University; Philip Tonkins, MS, DrPH; Peter Tugwell, MD; University of Ottawa; John Ware, QualityMetric Incorporated; Kevin Weinfurt, PhD; Duke University; Gordon Willis, PhD, National Cancer Institute; James P. Witter, MD, PhD NIAMS; Albert Wu, MD, MPH; Susan Yount, PhD, Northwestern University.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Magasi, S., Ryan, G., Revicki, D. et al. Content validity of patient-reported outcome measures: perspectives from a PROMIS meeting. Qual Life Res 21, 739–746 (2012). https://doi.org/10.1007/s11136-011-9990-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-011-9990-8

Keywords

Navigation