The recent Cancer Strategy for England (2016) highlights that by March 2017 NHS England will have agreed an approach for data collection which includes patient-reported outcome measures (PROMs) as a means for assessing long-term quality of life (QoL) for cancer patients . Moreover, it indicates that “people affected by cancer (and clinical leaders) … will be in the driving seat for improving quality across cancer pathways” ( p. 18).
For this to work and be sustained requires all stakeholders and end-users—including policy makers, health professionals, methodologists and patients—to contribute to the co-construction of a process (including PROM selection) that is relevant and fit for purpose. Several key questions must drive the process: what do the different stakeholders need from the QoL data? (what to measure?); how will these needs be reflected in the choice of PROM? (how to measure?); when should QoL be assessed? (when to measure?); and how can agreement between stakeholders be achieved? (what does consensus look like?). Additionally, consideration must be given to the education and support required to support data interpretation (what does the data mean?). Strong collaborative relationships between key players will be crucial throughout the process, from question definition to implementation, to ensure buy-in for the recommendations  and to ensure consistency in the way data are interpreted.
What and When to Measure and Achieving Consensus
Historically, the defining of outcomes has largely been driven by the perspectives of health professionals and methodologists. For example, a recent review of clinical trials in resuscitation science highlighted the (not surprising) focus on short-term survival and clinician-reported outcome, but also the (surprising) failure to include the survivors’ perspective or longer-term assessment of QoL . While such short-term outcomes are of clear importance to clinicians and healthcare providers, they lack both relevance and trajectory for patients. For example, how meaningful is a clinician-based assessment of function at hospital discharge? What is important to patients—in both seeking to communicate ‘how well they survive’ and in seeking to understand the benefits (and side effects) of treatment—is their quality of life over time, not at a solitary point in time (particularly when so much changes following hospital discharge).
Growing awareness of the discrepancies that exist between outcomes defined by a non-patient population versus those defined by patients living with a health condition has resulted in a move towards the greater involvement of patients in the identification of important outcomes [4–6]. The importance (and challenges) of engaging with multiple perspectives—including patients—in defining what to measure in clinical trials and routine practice settings is increasingly evident in the development of core outcome sets (COS)  (http://www.comet-initiative.org). Achieving consensus in outcome reporting (defining a common standard for outcomes for specific conditions and across specific settings) reduces variation in outcome reporting and supports data comparison across clinical trials, research and registry databases, and routine practice [7–9]. In the era of ‘Big Data’ it also supports data linkage, providing evidence that, for example, can contribute to a developing picture of different cancer pathways [10, 11]. A clear and accepted standard for what must be reported also reduces concerns over potential reporting bias [12, 13]: if the outcome has been clearly defined, the results should be analysed consistently and communicated transparently.
The 2016 Cancer Strategy describes a Cancer Dashboard that will include a long-term QoL metric that will “serve as the single version of the truth on cancer outcomes” ( p. 18). Clarity in the aspects of QoL that are important is therefore crucial—and essential—to informing the choice of metric or method of assessment (how to measure). QoL is a multi-faceted, broad-ranging construct, including both health-related and non-health-related concepts . What ‘quality of life’ means to a clinician—and indeed, what information the clinician needs from this information—may differ significantly to the way in which QoL is defined by a patient or by a healthcare provider, and the information that they need to inform their individual decision making about access to or provision of that treatment. Hence, the acceptability and validity of the metric which purports to measure QoL may differ between stakeholders.
How to Measure and Understand the Data
Once the question of ‘what to measure’ has been resolved, a method of assessment is required. The growing focus on seeking to better understand the patient perspective, coupled with developments in measurement science, have resulted in significant growth in the availability of PROMs . Well developed PROMs are questionnaires, often containing multiple items, which seek to provide a patient-derived assessment of how patients feel; what they can and cannot do, both physically and psychosocially; and hence how well they live their life, as a consequence of their health and associated healthcare. They should, ideally, include the outcomes that really matter to patients. However, not all PROMs do this equally well.
First, PROMs may be simply classified as generic (broad ranging and suitable for completion by the general population and population groups) or specific [15, 16]. Specific measures may be specific to a condition (e.g. breast cancer), a population (e.g. children), an aspect of health (e.g. fatigue), or to an intervention (e.g. hip replacement). While generic measures may lack specificity, they are important in supporting health-state comparisons across population and patient groups . Moreover, several measures—such as the ubiquitous, generic, preference-based measure, the EuroQol EQ-5D (, https://www.euroqol.org/about-eq-5d.html)—contribute to cost-effectiveness analyses and are widely used in Health Technology Assessments (HTAs) to inform NICE decision making [18, 19]. Well developed specific measures should be more relevant to patient groups than their generic counterparts, and hence better able to detect the changes in health that really matter [15, 20]. Good practice guidance suggests that generic and specific PROMs provide complementary information and that both should be used in healthcare assessment .
Second, the crafting of PROMs has, historically, been largely driven by the perspectives of health professionals and methodologists. While the resulting ‘legacy’ measures such as the EQ-5D often have acceptable measurement properties in certain population groups [19, 20], for many end-users such measures present a ‘crude’ illustration of the impact of an illness and have a limited role to play in supporting individual decision making . Moreover, where there is a perception that measures have been poorly developed and decision making is based upon evidence from flawed tools, their application may evoke a degree of hostility, disbelief and/or resistance amongst health professionals, payors and patients [5, 21].
A growing awareness of the limitations of existing PROMs and discrepancies that exist between the outcomes defined by patients living with a health condition and those defined by others, has resulted in a move towards the greater involvement of patients as participants or collaborators in PROM development and selection [5, 22–24]. This can be witnessed in the greater transparency and auditable contribution of patients as participants in PROM development—for example, in qualitative research to inform the conceptual underpinning, associated item generation and use of language that resonates with patients [24, 25]. This approach seeks to improve the relevance, face and content validity of measures  and has been embraced by major regulatory authorities and HTA agencies, in particular the FDA  and NICE [18, 19].
The move towards the greater involvement of patients as research partners or active collaborators seeks to ensure that patients are co-drivers through all stages of PROM development, selection, implementation and evaluation [5, 22, 27–29]. The resulting co-production, or co-selection, of a PROM seeks to ensure that the measure has greater resonance with all stakeholders, the data is taken seriously and that it is, indeed, fit for purpose. Moreover, growing evidence highlights that patient involvement has its greatest value when structured early in study development, and its least when added as if an afterthought [2, 22, 23, 26, 29]; this was recognised early in the development of the National Institute for Health Research (NIHR) .
A further essential consideration is data interpretation. The qualitative meaning of PROM scores is not intuitively apparent , and ensuring that PROMs are used to their best advantage requires additional support for data interpretation. Two statistically generated values are important in this context: the smallest detectable change (SDC)—a change that is greater than measurement error; and the minimal important change (MIC)—the smallest change in score that patients perceive as important [31, 32]. Determination of these values is often used to underpin the graphic illustration of scores—and more specifically change in scores—supporting data interpretation. Working together with patients and/or health professionals, there are examples of embedding PROMs within e-health systems and linking the interpretation of change scores to ‘traffic lights’ , to clinical vignettes and decision trees .
With patients firmly positioned as co-drivers, the time has come for PROMs to move into the fast lane of healthcare. Evidence of the scientific rigour of PROMs as measures of explicit, meaningful variables is essential to ensuring that PROM data is high quality [15, 21, 32]. To ensure this rigour, there have been calls for clinicians to be formally trained in the science of health measurement . We now suggest a further evolution to this call—the need for more patients to be formally involved as research partners throughout all stages of PROM co-construction, selection and implementation of PROMs .
But, PROMs are not the answer to everything, and provide only a patient’s perspective of the ‘truth’. It is important that we understand what PROMs can and cannot do. High quality, relevant and acceptable PROMs will enhance our understanding of the impact of ill-health, raise our awareness of patient needs, and hence inform the provision of more tailored health and social care, with the ultimate goal of improving health and well-being. Moreover, well developed PROMs should facilitate dialogue and communication between health professionals and patients, leading to better joint decision making moving forward [34–36].
However, PROMs are not intended to be used in isolation or indeed to replace clear clinical judgement; their judicious integration into practice must recognise the philosophical underpinning of patient-reported assessment and its role within patient-centred care. England’s Cancer Strategy 2016 must seek to facilitate the transparent integration of patient values captured within a PROM, with appropriate research evidence and clinical expertise to enable a complete understanding of the truth on cancer outcomes, and hence the evidence to underpin access to new medicines. For those conditions where less attention has been afforded to seeking to better understand the outcomes that really matter to patients, a similar model of integration and co-construction which seeks to make the truth about outcomes more transparent is recommended.
Achieving World-Class Cancer Outcomes: Taking the strategy forward. Five Year Forward View. 2016. https://www.england.nhs.uk/wp-content/uploads/2016/05/cancer-strategy.pdf. Accessed 28 July 2016.
ReseArch with Patient and Pulich involvement: a RealisT evaluation—the RAPPORT study. Health Services and Delivery Research, vol 3, Issue 38. 2015. ISSN 2050-4349. http://www.journalslibrary.nihr.ac.uk/__data/assets/pdf_file/0009/152298/FullReport-hsdr03380.pdf.
Whitehead L, Perkins GD, Clarey A, Haywood KL. A systematic review of the outcomes reported in cardiac arrest clinical trials: the need for a core outcome set. Resuscitation. 2015;88:150–7.
Hewlett SA. Patients and clinicians have different perspectives on outcomes in arthritis. J Rheumatol. 2003;30(4):877–9.
Staniszewska S, Haywood KL, Brett J, Tutton L. Patient and public involvement in patient-reported outcome measures: evolution not revolution. Patient. 2012;5(2):79–87.
Gough NJ, Smith C, Ross JR, Riley J, Judson I. Symptom burden, survival and palliative care in advanced soft tissue sarcoma. Sarcoma. 2011;2011. Article ID 325189 (Epub Dec 2011).
Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, Tugwell P. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132.
Schmitt J, Langan S, Stamm T, Williams HC; Harmonizing Outcome Measurements in Eczema (HOME) Delphi panel. Core outcome domains for controlled trials and clinical recordkeeping in eczema: international multiperspective Delphi consensus process. J Invest Dermatol. 2011;131(3):623–30.
Haywood KL, Whitehead L, Perkins GD. The psychosocial outcomes of cardiac arrest: relevant and robust patient-centred assessment is essential. Resuscitation. 2014;85(6):718.
Murdoch TB, Detsky AS. The inevitable application of big data to health care. JAMA. 2013;309(13):1351–2.
Wilson R. Use MY data mini blog. 2016. http://www.usemydata.org/blog1.shtml. Accessed 28 July 2016.
Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, Williamson PR. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365.
Smith V, Clarke M, Williamson P, Gargon E. Survey of new 2007 and 2011 Cochrane reviews found 37 % of prespecified outcomes not reported. J Clin Epidemiol. 2015;68(3):237–45.
Huber M, Knottnerus JA, Green L, van der Horst H, Jadad AR, Kromhout D, Leonard B, Lorig K, Loureiro MI, van der Meer JW, Schnabel P, Smith R, van Weel C, Smid H. How should we define health? BMJ. 2011;343:d4163.
Garratt A, Schmidt L, Mackintosh A, Fitzpatrick R. Quality of life measurement: bibliographic study of patient assessed health outcome measures. BMJ. 2002; 324(7351):1417.
Haywood KL, Staniszewska S, Chapman S. Quality and acceptability of patient-reported outcome measures used in chronic fatigue syndrome/myalgic encephalomyelitis (CFS/ME): a systematic review. Qual Life Res. 2012;21(1):35–52.
EuroQoL Group. EuroQol—a new facility for the measurement of health-related quality of life. Health Policy. 1990;16(3):199–208. http://www.euroqol.org/about-eq-5d.html. Accessed 28 July 2016.
National Institute for Health and Care Excellence (NICE). Guide to the methods of technology appraisal 2013. Process and methods guides. London: National Institute of Health and Clinical Excellence. 2013. https://www.nice.org.uk/article/pmg9/resources/non-guidance-guide-to-the-methods-of-technology-appraisal-2013-pdf. Accessed 28 July 2016.
Longworth L, Yang Y, Young T. Use of generic and condition-specific measures of health-related quality of life in NICE decision-making: a systematic review, statistical modelling and survey. Health Technol Assess. 2014;18(9). ISSN 1366-5278.
Devlin N, Appleby J. Getting the Most Out of PROMs. Putting health outcomes at the heart of NHS decision-making. The Kings Fund, London. 2010. https://www.kingsfund.org.uk/sites/files/kf/Getting-the-most-out-of-PROMs-Nancy-Devlin-John-Appleby-Kings-Fund-March-2010.pdf. Accessed 28 July 2016.
Hobart JC, Cano SJ, Zajicek JP, Thompson AJ. Rating scales as outcome measures for clinical trials in neurology: problems, solutions, and recommendations. Lancet Neurol. 2007;6:1094–105.
de Wit M, Abma T, Koelewijn-van Loon M, Collins S, Kirwan J. Involving patient research partners has a significant impact on outcomes research: a responsive evaluation of the international OMERACT conferences. BMJ Open. 2013;3(5) (pii: e002241).
Gossec L, de Wit M, Kiltz U, Braun J, Kalyoncu U, Scrivo R, Maccarone M, Carton L, Otsa K, Sooäär I, Heiberg T, Bertheussen H, Cañete JD, Sánchez Lombarte A, Balanescu A, Dinte A, de Vlam K, Smolen JS, Stamm T, Niedermayer D, Békés G, Veale D, Helliwell P, Parkinson A, Luger T, Kvien TK; EULAR PsAID Taskforce. A patient-derived and patient-reported outcome measure for assessing psoriatic arthritis: elaboration and preliminary validation of the Psoriatic Arthritis Impact of Disease (PsAID) questionnaire, a 13-country EULAR initiative. Ann Rheum Dis. 2014;73(6):1012–9.
US Food and Drug Administration Guidance for Industry: Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Rockville: Department of Health and Human Services, Food and Drug Administration, Centre for Drug Evaluation and Research; 2009.
Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, Ring L. Content validity–establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1–eliciting concepts for a new PRO instrument. Value Health. 2011;14(8):967–77.
de Wit MP, Kvien TK, Gossec L. Patient participation as an integral part of patient-reported outcomes development ensures the representation of the patient voice: a case study from the field of rheumatology. RMD Open. 2015;1(1):e000129.
Cheung PP, de Wit M, Bingham CO 3rd, Kirwan JR, Leong A, March LM, Montie P, Scholte-Voshaar M, Gossec L. Recommendations for the Involvement of Patient Research Partners (PRP) in OMERACT Working Groups. A Report from the OMERACT 2014 Working Group on PRP. J Rheumatol. 2016;43(1):187–93.
Tillett W, Adebajo A, Brooke M, Campbell W, Coates LC, FitzGerald O, Gossec L, Helliwell P, Hewlett S, James J, Minnock P, Reast A, O’Sullivan D, de Wit M, McHugh N. Patient involvement in outcome measures for psoriatic arthritis. Curr Rheumatol Rep. 2014;16(5):418.
Frank L, Forsythe L, Ellis L, Schrandt S, Sheridan S, Gerson J, Konopka K, Daugherty S. Conceptual and practical foundations of patient engagement in research at the patient-centred outcomes research institute. QLRes. 2015;24:1033–41.
Stewart D, Wilson R, Selby P, Darbyshire J. Patient and public involvement. Ann Oncol. 2011;22(Suppl 7):vii54–56.
de Vet HC, Terwee CB, Ostelo RW, Beckerman H, Knol DL, Bouter LM. Minimal changes in health status questionnaires: distinction between minimally detectable change and minimally important change. Health Qual Life Outcomes. 2006;4:54.
de Vet HCW, Terwee CB, Mokkink L, Knol D. Measurement in medicine: a practical guide (Practical Guides to Biostatistics and Epidemiology). Cambridge: Cambridge University Press; 2011.
Ahmed S, Bartlett SJ, Ernst P, Paré G, Kanter M, Perreault R, Grad R, Taylor L, Tamblyn R. Effect of a web-based chronic disease management system on asthma control and health-related quality of life: study protocol for a randomized controlled trial. Trials. 2011;12:260.
Santana MJ1, Haverman L, Absolom K, Takeuchi E, Feeny D, Grootenhuis M, Velikova G. Training clinicians in how to use patient-reported outcome measures in routine clinical practice. Qual Life Res. 2015;24(7):1707–18 (Epub 2015 Jan 15).
Haywood K, Marshall S, Fitzpatrick R. Patient participation in the consultation process: a structured review of intervention strategies. Patient Educ Couns. 2006;63(1–2):12–23.
Greenhalgh J, Long AF, Flynn R. The use of patient reported outcome measures in routine clinical practice: lack of impact or lack of theory? Soc Sci Med. 2005;60(4):833–43.
This editorial did not receive any financial support.
Conflict of interest
Author Kirstie L. Haywood is co-founder, past co-chair and current chair-elect of the International Society for Quality-of-Life Research (ISOQOL) Patient Engagement Special Interest Group. Author Roger Wilson declares that he has no conflict of interest. Author Sophie Staniszewska declares that she has no conflict of interest. Author Sam Salek is co-founder, past co-chair and current chair of the ISOQOL Patient Engagement Special Interest Group and has received Educational and Travel grants from the European Hematology Association, Shire Pharmaceuticals and Agios.
This editorial does not contain any studies with human participants or animals performed by any of the authors.
About this article
Cite this article
Haywood, K.L., Wilson, R., Staniszewska, S. et al. Using PROMs in Healthcare: Who Should Be in the Driving Seat—Policy Makers, Health Professionals, Methodologists or Patients?. Patient 9, 495–498 (2016). https://doi.org/10.1007/s40271-016-0197-5