Skip to main content
SpringerLink
Log in

Streamlining the Validation of Patient Reported Outcome (PRO) Measures in Drug Regulatory Processes

  • Current Opinion
  • Published:
Pharmaceutical Medicine Aims and scope Submit manuscript

Abstract

The context and setting in which patient-reported outcome (PRO) measures applied in drug development programmes are developed and applied are undergoing rapid changes. The emergence of social media, coupled with advancement of patient-centred methodologies, has led to more connected and empowered patients than before. Drug regulatory agencies are beginning to embrace a new role as collaborators in the drug development process, and new models for defining core outcomes in various disease areas are emerging. Technological advances and social changes have led to the wide use of personal gadgets with greater internet connectivity and computing power than most computers had a generation ago. To explore how progress in the measurement of PROs might be achieved amidst such changes, lessons were drawn from the PRO labelling claims reviewed by the US FDA following the publication of the PRO guidance in 2009. In addition, semi-structured interviews were carried out with seven experts working with drug regulatory agencies and in the pharmaceutical industry to identify major issues and possible solutions. The following recommendations are proposed: (1) that comprehensive hypotheses about the PRO concept be developed; (2) that PRO concepts/measures maintain substantive and clinical validity; (3) that the use of advanced psychometric methods be considered as the new gold standard in the development of PRO measures; (4) that psychometric properties of measures established through “validation by application” be enhanced; (5) that the generalizability of PRO measures be enhanced; (6) that the assessment of ability to detect change, and the interpretability of scores, be improved; and (7) that PRO guidelines by major regulatory authorities be aligned. These entail a greater transparency on the decisions taken during the PRO measure development process and greater understanding of the integration between qualitative aspects of the PRO measure and the score/measurement attributes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. The EXACT is a daily diary for evaluating acute exacerbations of COPD and chronic bronchitis (AECB) [19]. No labelling claim has been granted based on the EXACT to date following its qualification by the FDA.

References

  1. US Food and Drug Administration. US Food and Drug Administration guidance for industry-patient reported outcome measures: use in medical product development to support labelling claims; 2009. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf. Accessed 15 Jan 2012.

  2. Ahmed S, Berzon RA, Revicki DA, Lenderking WR, Moinpour CM, Basch E, et al. The use of patient-reported outcomes (PRO) within comparative effectiveness research: implications for clinical practice and health care policy. Med Care. 2012;50(12):1060–70.

    Article  PubMed  Google Scholar 

  3. Snyder CF, Aaronson NK, Choucair AK, Elliott TE, Greenhalgh J, Halyard MY, et al. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res. 2012;21(8):1305–14.

    Article  PubMed  Google Scholar 

  4. European Medicines Agency. Reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal products. London: European Medicines Agency, Committee for Medicinal Products for Human Use; 2005.

    Google Scholar 

  5. Marquis P, Caron M, Emery M-P, Scott JA, Arnould B, Acquadro C. The role of health-related quality of life data in the drug approval processes in the US and Europe. Pharm Med. 2011;25(3):147–60.

    Article  Google Scholar 

  6. Baldwin MM, Spong A, Doward L, Gnanasakthy A. Patient-reported outcomes, patient-reported information. Patient. 2011;4(1):11–7.

    Article  PubMed Central  PubMed  Google Scholar 

  7. Frost J, Okun S, Vaughan T, Heywood J, Wicks P. Patient-reported outcomes as a source of evidence in off-label prescribing: analysis of data from PatientsLikeMe. J Med Internet Res. 2011;13(1):e6.

    Article  PubMed Central  PubMed  Google Scholar 

  8. Bender JL, Jimenez-Marroquin M-C, Jadad AR. Seeking support on facebook: a content analysis of breast cancer groups. J Med Internet Res. 2011;13(1):e16.

    Article  PubMed Central  PubMed  Google Scholar 

  9. Hamm MP, Chisholm A, Shulhan J, Milne A, Scott SD, Given LM, et al. Social media use among patients and caregivers: a scoping review. BMJ Open. 2013;3(5). doi:10.1136/bmjopen-2013-002819.

  10. Tweet MS, Gulati R, Aase LA, Hayes SN. Spontaneous coronary artery dissection: a disease-specific, social networking community-initiated study. Mayo Clin Proc. 2011;86(9):845–50.

    Article  PubMed Central  PubMed  Google Scholar 

  11. DiBenedetti DB, Coles TM, Sharma T. Recruiting patients with a rare blood disorder and their caregivers through social media. Value Health. 2013;16(3):A51.

    Article  Google Scholar 

  12. Gustafson DL, Woodworth CF. Methodological and ethical issues in research using social media: a metamethod of human papillomavirus vaccine studies. BMC Med Res Methodol. 2014;14(1):127.

    Article  PubMed Central  PubMed  Google Scholar 

  13. Rothman M, Gnanaskathy A, Wicks P, Papadopoulos EJ. Can we use social media to support content validity of patient-reported outcome instruments in medical product development? Value Health. 2015;18(1):1–4.

    Article  PubMed  Google Scholar 

  14. Kamudoni P, Mueller B, Salek M. The development and validation of a disease-specific quality of life measure in hyperhidrosis: the Hyperhidrosis Quality of Life Index (HidroQOL©). Qual Life Res. 2015;24(4):1017–27.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  15. European Medicines Agency. European Medicines Agency Guidance for applicants seeking scientific advice and protocol assistance. London: European Medicines Agency; 2014.

    Google Scholar 

  16. European Medicines Agency. Best practice guidance for Pilot EMA HTA Parallel Scientific Advice procedures; 2014. European Medicines Agency.

  17. European Medicines Agency. EMA/CHMP/SAWP/178465/2015. Draft qualification opinion of qualification of exacerbations of chronic pulmonary disease tool (EXACT), and EXACT- respiratory symptoms measure (E-RS) for evaluating treatment outcomes in clinical trials in COPD. London: European Medicines Agency; 2015.

  18. US Food and Drug Administration. Draft guidance for industry on qualification of exacerbations of chronic pulmonary disease tool for measurement of symptoms of acute bacterial exacerbation of chronic bronchitis in patients with chronic obstructive pulmonary disease. Federal Register. 2014;79(7).

  19. Leidy NK, Murray LT. Patient-reported outcome (PRO) measures for clinical trials of COPD: the EXACT and E-RS. COPD J Chron Obstr Pulmon Dis. 2013;10(3):393–8.

    Article  Google Scholar 

  20. Boers M, Kirwan JR, Wells G, Beaton D, Gossec L, d’Agostino MA, et al. Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0. J Clin Epidemiol. 2014;67(7):745–53. doi:10.1016/j.jclinepi.2013.11.013.

    Article  PubMed  Google Scholar 

  21. Rudwaleit M, van der Heijde D, Landewé R, Listing J, Akkoc N, Brandt J, et al. The development of Assessment of SpondyloArthritis international Society classification criteria for axial spondyloarthritis (part II): validation and final selection. Ann Rheum Dis. 2009;68(6):777–83. doi:10.1136/ard.2009.108233.

    Article  CAS  PubMed  Google Scholar 

  22. Coons S, Kothari S, Monz B, Burke L. The patient-reported outcome (PRO) consortium: filling measurement gaps for PRO end points to support labeling claims. Clin Pharmacol Ther. 2011;90(5):743–8.

    Article  CAS  PubMed  Google Scholar 

  23. Leidy NK, Murray LT. Patient-reported outcome (PRO) measures for clinical trials of COPD: the EXACT and E-RS. COPD. 2013;10(3):393–8. doi:10.3109/15412555.2013.795423.

    Article  PubMed  Google Scholar 

  24. Doward L, Spoorenberg A, Cook S, Whalley D, Helliwell P, Kay L, et al. Development of the ASQoL: a quality of life instrument specific to ankylosing spondylitis. Ann Rheum Dis. 2003;62(1):20–6.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  25. Reeve BB, Mâsse LC. Item response theory modeling for questionnaire evaluation. In: Presser S, Rothgeb JM, Couper MP, Lessler JT, Martin E, Martin J, et al., editors. Methods for testing and evaluating survey questionnaires. Hoboken: Wiley; 2004. p. 247–73.

    Chapter  Google Scholar 

  26. Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care. 2007;45(5 Suppl 1):S3.

    Article  PubMed Central  PubMed  Google Scholar 

  27. Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. J Clin Epidemiol. 2010;63(11):1179–94.

    Article  PubMed Central  PubMed  Google Scholar 

  28. Rose M, Fischer F, Nolte S. Who is the norm? Hurdles on the way to an international standardisation of PRO assessments. Qual Life Res. 2014;23(Suppl. 1):12.

    Google Scholar 

  29. Symonds T. Sharing the progress of the PROMIS industry interest committee. Qual Life Res. 2014;23(Supplement 1):12.

    Google Scholar 

  30. Eremenco S, Coons SJ, Paty J, Coyne K, Bennett AV, McEntegart D. PRO data collection in clinical trials using mixed modes: report of the ISPOR PRO mixed modes good research practices task force. Value Health. 2014;17(5):501–16. doi:10.1016/j.jval.2014.06.005.

    Article  PubMed  Google Scholar 

  31. Coons S, Eremenco S, Lundy JJ, O’Donohoe P, O’Gorman H, Malizia W. Capturing patient-reported outcome (PRO) data electronically: the past, present, and promise of ePRO measurement in clinical trials. Patient. 2014;8:301–9. doi:10.1007/s40271-014-0090-z.

    Article  PubMed Central  Google Scholar 

  32. Yeomans A. The future of ePRO platforms. Applied Clinical Trials; 2014.

  33. Gnanasakthy A, Lewis S, Clark M, Mordin M, DeMuro C. Potential of patient-reported outcomes as nonprimary endpoints in clinical trials. Health Qual Life Outcomes. 2013;11:83. doi:10.1186/1477-7525-11-83.

    Article  PubMed Central  PubMed  Google Scholar 

  34. Gnanasakthy A, DeMuro C, Clark M, Mordin M, Thomas S. Role of patient-reported outcome measures in the assessment of central nervous system agents. Ther Innov Regul Sci. 2013;47(5):613–8.

    Article  Google Scholar 

  35. Gnanasakthy A, Mordin M, Clark M, DeMuro C, Fehnel S, Copley-Merriman C. A review of patient-reported outcome labels in the United States: 2006 to 2010. Value Health. 2012;15(3):437–42.

    Article  PubMed  Google Scholar 

  36. Hao Y. Patient-reported outcomes in support of oncology product labeling claims: regulatory context and challenges. Expert Rev Pharmacoecon Outcomes Res. 2010;10(4):407–20.

    Article  PubMed  Google Scholar 

  37. DeMuro C, Clark M, Mordin M, Fehnel S, Copley-Merriman C, Gnanasakthy A. Reasons for rejection of patient-reported outcome label claims: a compilation based on a review of patient-reported outcome use among new molecular entities and biologic license applications, 2006–2010. Value Health. 2012;15(3):443–8. doi:10.1016/j.jval.2012.01.010.

    Article  PubMed  Google Scholar 

  38. Patrick DL, Burke LB, Powers JH, Scott JA, Rock EP, Dawisha S, et al. Patient-reported outcomes to support medical product labeling claims: FDA perspective. Value Health. 2007;10:S125–37.

    Article  PubMed  Google Scholar 

  39. Rothman ML, Beltran P, Cappelleri JC, Lipscomb J, Teschendorf B, Mayo/FDA Patient-Reported Outcomes Consensus Meeting Group. Patient-reported outcomes: conceptual issues. Value Health . 2007;10:S66–75. doi:10.1111/j.1524-4733.2007.00269.x.

    Article  PubMed  Google Scholar 

  40. Leidy NK, Vernon M. Perspectives on patient-reported outcomes : content validity and qualitative research in a changing clinical trial environment. Pharmacoeconomics. 2008;26(5):363–70.

    Article  PubMed  Google Scholar 

  41. Streiner DL, Norman GR. Health Measurement Scales: a practical guide to their development and use. Oxford: Oxford University Press; 2008.

    Book  Google Scholar 

  42. Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, et al. Content validity—establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 2—assessing respondent understanding. Value Health. 2011;14(8):978–88.

    Article  PubMed  Google Scholar 

  43. Sjodahl Hammarlund C, Nilsson MH, Idvall M, Rosas SR, Hagell P. Conceptualizing and prioritizing clinical trial outcomes from the perspectives of people with Parkinson’s disease versus health care professionals: a concept mapping study. Qual Life Res. 2014;23(6):1687–700. doi:10.1007/s11136-013-0614-3.

    Article  PubMed  Google Scholar 

  44. Haywood K, Brett J, Salek S, Marlett N, Penman C, Shklarov S, et al. Patient and public engagement in health-related quality of life and patient-reported outcomes research: what is important and why should we care? Findings from the first ISOQOL patient engagement symposium. Qual Life Res. 2014;24:1069–76. doi:10.1007/s11136-014-0796-3.

    Article  PubMed  Google Scholar 

  45. Forsythe LP, Szydlowski V, Murad MH, Ip S, Wang Z, Elraiyah TA, et al. A systematic review of approaches for engaging patients for research on rare diseases. J Gen Intern Med. 2014;29(Suppl 3):788–800. doi:10.1007/s11606-014-2895-9.

    Article  PubMed Central  Google Scholar 

  46. Magasi S, Ryan G, Revicki D, Lenderking W, Hays RD, Brod M, et al. Content validity of patient-reported outcome measures: perspectives from a PROMIS meeting. Qual Life Res. 2011;21:739–46.

    Article  PubMed  Google Scholar 

  47. Thomas ML. The value of item response theory in clinical assessment: a review. Assessment. 2011;18(3):291–307.

    Article  PubMed  Google Scholar 

  48. Thompson JR, Cappelleri JC, Getter C, Pleil A, Reichel M, Wolf S. Enhanced interpretation of instrument scales using the Rasch model. Drug Inf J. 2007;41(4):541–50.

    Google Scholar 

  49. Izem R, Kammerman LA, Komo S. Statistical challenges in drug approval trials that use patient-reported outcomes. Stat Methods Med Res. 2013;. doi:10.1177/0962280213476376.

    PubMed  Google Scholar 

  50. Byrne B. Structural equation modeling with Mplus: basic concepts, applications, and programming. New York: Routledge; 2011.

    Google Scholar 

  51. Rothman M, Burke L, Erickson P, Leidy NK, Patrick DL, Petrie CD. Use of existing patient-reported outcome (PRO) instruments and their modification: the ISPOR good research practices for evaluating and documenting content validity for the use of existing instruments and their modification PRO task force report. Value Health. 2009;12(8):1075–83.

    Article  PubMed  Google Scholar 

  52. Reise SP, Ainsworth AT, Haviland MG. Item response theory fundamentals, applications, and promise in psychological research. Curr Dir Psychol Sci. 2005;14(2):95–101.

    Article  Google Scholar 

  53. Terwee C, Dekker F, Wiersinga W, Prummel M, Bossuyt P. On assessing responsiveness of health-related quality of life instruments: guidelines for instrument evaluation. Qual Life Res. 2003;12(4):349–62.

    Article  CAS  PubMed  Google Scholar 

  54. de Vet HC, Bouter LM, Bezemer PD, Beurskens AJ. Reproducibility and responsiveness of evaluative outcome measures. Int J Technol Assess Health Care. 2001;17(04):479–87.

    PubMed  Google Scholar 

  55. Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol. 2008;61(2):102–9.

    Article  PubMed  Google Scholar 

  56. Gerlinger C, Schmelter T. Determining the non-inferiority margin for patient reported outcomes. Pharm Stat. 2011;10(5):410–3. doi:10.1002/pst.507.

    Article  PubMed  Google Scholar 

  57. McLeod DS, Warner JV, Henman M, Cowley D, Gibbons K, McIntyre HD. Associations of serum vitamin D concentrations with obstetric glucose metabolism in a subset of the Hyperglycemia and Adverse Pregnancy Outcome (HAPO) study cohort. Diabet Med. 2012;29(8):e199–204. doi:10.1111/j.1464-5491.2011.03551.x.

    Article  CAS  PubMed  Google Scholar 

  58. de Vet HC, Ostelo RW, Terwee CB, van der Roer N, Knol DL, Beckerman H, et al. Minimally important change determined by a visual method integrating an anchor-based and a distribution-based approach. Qual Life Res. 2007;16(1):131–42.

    Article  PubMed Central  PubMed  Google Scholar 

  59. Lohr KN. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res. 2002;11(3):193–205. doi:10.1023/a:1015291021312.

    Article  PubMed  Google Scholar 

  60. Leidy NK, Revicki DA, Genesté B. Recommendations for evaluating the validity of quality of life claims for labeling and promotion. Value Health. 1999;2(2):113–27.

    Article  CAS  PubMed  Google Scholar 

  61. Frost MH, Reeve BB, Liepa AM, Stauffer JW, Hays RD, Mayo/FDA Patient-Reported Outcomes Consensus Meeting Group. What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value Health. 2007;10(Suppl. 2):S94–105. doi:10.1111/j.1524-4733.2007.00272.x.

    Article  PubMed  Google Scholar 

  62. Wyrwich KW, Bullinger M, Aaronson N, Hays RD, Patrick DL, Symonds T. Estimating clinically significant differences in quality of life outcomes. Qual Life Res. 2005;14(2):285–95.

    Article  PubMed  Google Scholar 

  63. Salek M, Kamudoni P. Quality of life measurement in dermatology consultation: impact on patient reported outcomes. Giornale italiano di dermatologia e venereologia: organo ufficiale, Societa italiana di dermatologia e sifilografia. 2013;148(3):263–75.

    CAS  Google Scholar 

Download references

Acknowledgments

We gratefully acknowledge the support of the seven experts working with drug regulatory agencies and in the pharmaceutical industry who agreed to be interviewed as part of the research for this manuscript.

Author information

Authors and Affiliations

  1. Department of Pharmacy, School of Life and Medical Sciences, University of Hertfordshire, College Lane, Hatfield, Herts, UK

    Sam Salek

  2. Institute for Medicines Development, Cardiff, UK

    Sam Salek & Paul Kamudoni

Authors
  1. Sam Salek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul Kamudoni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sam Salek.

Ethics declarations

Funding

No funding was received for the preparation or publication of this article.

Conflict of interest

Professor Salek and Dr. Kamudoni have no conflicts of interest to declare.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (PDF 164 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Salek, S., Kamudoni, P. Streamlining the Validation of Patient Reported Outcome (PRO) Measures in Drug Regulatory Processes. Pharm Med 29, 255–268 (2015). https://doi.org/10.1007/s40290-015-0110-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40290-015-0110-x

Keywords

Search

Navigation