FormalPara Infographic
figure a

Digital Features for this article can be found at https://doi.org/10.6084/m9.figshare.14562672.

FormalPara Key Points

We developed a brief PRO instrument to measure whether patients in a clinical trial think that ‘good’ experiences outweigh ‘bad’ experiences with the study medication.

Concept elicitation interviews across disease areas showed that patients considered effectiveness of the treatment, burden of side effects, convenience of use and overall acceptance of and satisfaction with the treatment when weighing the benefits against the risks of a new medicine.

1 Introduction

Benefit–risk assessment is key to medical treatment decision making. Listening directly to patients at all stages of drug design, development and use is critical to ensure that their needs are reflected in their healthcare. Understanding how individual patients think about the positive outcomes and negative consequences of their treatment adds an important additional aspect to population-level information from clinical trials [1]. Patient-perceived benefit–risk data have value for patients themselves and for stakeholders making decisions around drug development, regulatory assessments and reimbursement analyses [1,2,3,4,5]. In clinical practice, understanding the patient perspective of benefits and risks of therapies can help to optimize care and enhance adherence to treatment [1].

There is a need to understand what patients think about when they are asked to evaluate their treatment experience to enable researchers to generate hypotheses about how patients prioritize and weigh up perceived benefits and risks [1]. Such knowledge would serve as a foundation from which to develop methodologies for capturing patient-level benefit–risk evaluation appropriately. Benefit–risk assessments based on clinical study efficacy and safety data form part of regulatory submissions for new treatments [2, 4]. Supplementing the results of these assessments with patient-perceived benefit–risk information would ensure that the patient viewpoint was taken into account as part of regulatory decision making.

The drive to incorporate patient-centric benefit–risk assessments into clinical development and approval processes is recognized by regulatory bodies, including the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA) [1,2,3,4,5]. Submissions for regulatory assessment of a new medical treatment can be supported by acceptability of the treatment from the patient perspective [2, 4]. Both the FDA and the EMA consider information on patient attitudes and preferences regarding benefits, risks and the therapeutic context when assessing regulatory submissions [2, 4]. Weighing key benefits and key risks involves evaluating how the characteristics of the disease or medical condition to be treated (e.g. seriousness, chronicity, frequency) and the expected benefit of the new medicinal product compare with treatment risks (e.g. likelihood of occurrence, severity, ability to manage) and the availability and characteristics of treatment alternatives [2, 4].

Patient-reported outcome (PRO) questionnaires (‘instruments’) are commonly used in clinical trials to capture patient information fully and systematically. PRO questionnaire development adheres to strict methodological guidelines to ensure that the final instrument has content validity and is fit for purpose in the intended target population. Best practices for developing PRO instruments have been established [6,7,8,9]. A central aspect of good practice in PRO development is for instruments to be patient driven. Involvement of the target patient population is essential at an early stage of PRO questionnaire development. The foundation of instrument development consists of qualitative interviews with patients to elicit key aspects (‘concepts’) relevant and important to the target population(s) that should be measured by the questionnaire. For benefit–risk assessment, a valid and reliable patient-driven, patient-centric PRO instrument needs to capture what individuals normally think about in terms of benefit–risk and whether they consider that the ‘good’ of a treatment outweighs the ‘bad’.

The current study had two aims. The first was to gain an understanding of what patients think about when they evaluate their experience receiving treatment. The second was to develop a concise PRO instrument for application across disease areas to assess the patient’s overall view of benefit–risk of a new medicine in the intended context of use; that is, to measure whether patients in a clinical trial think that the ‘good’ outweighed the ‘bad’ experiences with the treatment. This study lays the groundwork for further psychometric evaluation and development of a scoring mechanism to provide an interpretable benefit–risk metric.

2 Methods

This study involved a literature review and concept elicitation interviews with patients to identify all relevant aspects of benefit–risk evaluation, drafting of questionnaire items and responses based on elicited concepts, and testing of the questionnaire via cognitive patient interviews, with iterative revisions to address feedback (Fig. 1). Interviews with patients were grouped into six sets (‘waves’) to allow for review of the results and changes to the interview guide and/or questionnaire between waves. The first interview wave was conducted for concept elicitation only, interviews in waves 2–4 contained both concept elicitation and cognitive debriefing components, and the last two waves of interviews comprised cognitive debriefing only. Best practices for establishing PRO instrument content validity were followed [6,7,8,9].

Fig. 1
figure 1

Overview of the study. PROQOLID Patient-Reported Outcome and Quality of Life Instruments Database

Full size image

2.1 Literature Review

The following sources of patient-assessed risk–benefit information were reviewed: regulatory guidance from the FDA, including the Center for Devices and Radiological Health, and EMA; forums and presentations from the International Society for Pharmacoeconomics and Outcomes Research, Pharmacoepidemiological Research on Outcomes of Therapeutics by a European Consortium and the Drug Information Association [10,11,12]; and existing PRO measures including the Treatment Satisfaction Questionnaire for Medication (TSQM) and the Patient Global Impression of Change [13, 14]. The Patient-Reported Outcome and Quality of Life Instruments Database (Mapi Research Trust) was searched for existing patient-reported benefit–risk instruments.

2.2 Patient Interviews

2.2.1 Design and Participants

Concept elicitation and cognitive debriefing interviews were conducted in a non-interventional, one-on-one qualitative study in the USA. To be eligible for participation in the study, patients had to be aged 18 years or older and be receiving treatment for an oncological, respiratory, metabolic or cardiac condition—the intent was to include individuals with a variety of primary diseases (the key medical condition that most defined their life) to ensure applicability of the questionnaire across a range of therapeutic areas. Diversity was also sought within each wave of interviews with respect to health condition, age and sex. The first wave of interviews focused on eliciting concepts relating to the positive and negative aspects of treatment broadly in a real-world setting; thus, only patients who had not participated in a clinical trial were eligible. Subsequent waves of interviews focused on ensuring that the instrument would be fit for purpose in the intended context of use and included only patients with clinical trial experience.

Potential participants were identified via an online patient community (Mediguard), a clinical trial website (Accel Clinical Sites) and a market research company (Rare Patient Voice). One patient with cancer was recruited through AstraZeneca’s Patient Partnership Program. Researchers contacted patients via email (Mediguard, Rare Patient Voice) or telephone (Access Clinical Sites), and individuals who qualified through the screener and provided their consent form were asked to participate in an interview. Interviews were carried out by telephone with experienced research interviewers and lasted about 30–60 min.

The study was conducted in accordance with the Declaration of Helsinki ethical principles and Good Clinical Practice, and was approved by the New England Institutional Review Board. Concept elicitation and cognitive interviews were performed in accordance with regulatory guidelines [6, 9].

2.2.2 Concept Elicitation (Interview Waves 1–4)

Concept elicitation was performed during the first four waves of interviews using a semi-structured interview guide to explore how patients describe their treatment experience and decide whether a treatment is right for them. Open-ended questioning was used to encourage patients to report as much information as possible spontaneously without suggestions from the interviewer. Participants were prompted about pre-specified concepts identified from the literature review if these were not mentioned spontaneously during the open-ended portion of the interview. Patients in interview waves 2–4 were also asked open-ended questions about their experience with the medication received during the clinical trial; for example, ‘Thinking about your medication experience during the trial, what were some of the good things and bad things that you experienced while taking this [treatment]?’ and ‘Overall, did you think that this medication was worth taking?’. To assess the relative importance of concept categories, patients participating in the second and third waves of interviews were also asked to imagine allocating US$100 between identified concept categories according to each category’s value or importance, and to explain the rationale for the allocation.

Sociodemographic details were summarized using descriptive statistics. For the content and thematic analysis, a codebook was generated using inputs from the literature and concept elicitation interviews. De-identified transcripts from the first four waves of interviews were coded using ATLAS.ti software (Scientific Software Development GmbH, Berlin, Germany). For the first wave of interviews, which contained only a concept elicitation component, the entire transcripts were coded; for waves 2–4, which contained concept elicitation and cognitive components, coding was undertaken only for the concept elicitation sections. Concepts mentioned in response to open-ended questioning were coded as ‘spontaneous’ and those mentioned in response to prompts were coded as ‘probed’. Twenty percent of transcripts were coded by two coders and assessed for inter-coder agreement when both coders identified words or phrases reflecting the same code [7]. New concept codes were evaluated separately for each of the four interview waves. Concept saturation was assessed by comparing concepts identified in interview waves 2–4 with those in the previous wave(s), and was considered to be reached when no new concepts emerged from the interview process [7].

Draft items for a benefit–risk PRO instrument were created based on the key concept categories identified during concept elicitation that were considered specific to evaluating a medication received in a clinical trial. Wording of items and response options for a verbal rating scale were informed by the results of the concept elicitation interviews, review of other instruments such as the Patient Global Impression of Change and the authors’ prior experience, following best practices for item generation [8].

2.2.3 Cognitive Debriefing (Interview Waves 2–6)

Instrument items were debriefed during interview waves 2–6 to evaluate how each item would be interpreted by a patient in a clinical trial setting. Patients were asked to ‘think aloud’ during the item-by-item debrief. Probing questions were used during the item-by-item debrief if needed to supplement spontaneous reporting. Retrospective probing was used after completion of the instrument to ask about missing information. Items, instructions and responses were evaluated for clarity, relevance and appropriateness of response options, and were iteratively revised to incorporate patient feedback that could be tested in subsequent interview waves. If several patients recommended similar changes, or if a patient suggested or demonstrated the need for a significant change owing to lack of understanding of the wording used, potential edits were discussed by the IQVIA and AstraZeneca researchers. Once agreed, modifications were made to the instrument and an item matrix was used to document the changes, including when they occurred and their rationale. Instrument changes were submitted for approval by the Institutional Review Board. The final instrument was debriefed in the last interview wave.

3 Results

3.1 Literature Review

Guidance from regulatory and research bodies, and identified existing instruments, show that patients mix a broad set of concepts into their decision making when weighing up the benefits and risks of treatment, and that an individual’s perspective on benefit–risk trade-offs can vary according to their disease, demographics (e.g. age, sex) and psychological characteristics (e.g. personality) [11,12,13, 15,16,17,18,19]. Patients may better understand the idea of benefit–risk as whether or not they would ‘accept’ or ‘are satisfied with’ therapy [12, 13, 15, 16]. Four concepts for assessing benefit–risk of therapy from a patient perspective were identified: effectiveness of treatment, burden of side effects, convenience of use and overall acceptance/satisfaction/benefit–risk.

3.2 Patient Interviews

3.2.1 Sample Characteristics

A total of 47 patients (17 men and 30 women) participated in the study, with each wave of interviews including 5–10 patients. Table 1 shows the distribution of participant sex, age, highest education level and primary medical condition by interview wave. There was diversity within each wave of interviews regarding sociodemographic characteristics as well as primary medical condition. Overall, 20 patients had an oncological, 12 a respiratory, 10 a metabolic and five a cardiovascular primary medical condition.

Table 1 Patient demographics per interview wave
Full size table

3.2.2 Concept Elicitation

Thirty-two interviews, across waves 1–4, included a concept elicitation component. Patients described a broad range of concepts, almost all of which were captured spontaneously. Inter-coder agreement was 0.7 or higher for the 20% of transcripts that were coded by two coders. Concept saturation was met by the end of the second interview wave. Most concepts arose during wave 1; new concepts arising during wave 2 were related mainly to clinical trial participation and no new concepts arose in waves 3 or 4.

Concepts identified and the number of patients mentioning each concept, overall and according to whether mentions were spontaneous or probed, are listed in Table 2.

Table 2 Concepts for evaluating treatment experience and the number of patients mentioning them spontaneously (S), when probed (P) and in total (T) (total n = 32)
Full size table

3.3 Concept Categories

Elicited concepts could be grouped into seven categories. Over the course of the interviews, it became apparent that the broad range of concepts patients described included not only benefits and risks of the medication itself but also non-medication aspects of the clinical trial. Four concept categories were related directly to medication: effectiveness of treatment, burden of side effects, convenience of use and overall acceptance/satisfaction. Three additional categories captured the patient perspective of benefit–risk in relation to clinical trial participation that went beyond the medication itself: cost, trial experience and altruism.

3.3.1 Effectiveness of Treatment

Twenty-three patients (72%) talked about the effectiveness of treatment. It was considered the most important medication-related concept category across all primary medical conditions, irrespective of the severity of the disease/condition or the route of administration of therapy (oral, injection, infusion) (Fig. 2). When considering the ability of a medication to treat their condition, most patients viewed stabilization or improvement as a sign of effectiveness. Many patients also talked about the objective clinical measurement of a drug’s effectiveness.

Fig. 2
figure 2

Relative importance of concept categories assessed by asking patients participating in the second and third waves of interviews to imagine allocating US$100 between identified concept categories according to each category’s value or importance: a overall, b by route of therapy administration, c by primary medical condition and d by disease severity

Full size image

Patients with asymptomatic conditions, such as diabetes mellitus and some cancers, found quantitative measurements (e.g. blood test values) to be especially important indicators of effectiveness. Example quotations for the ability to treat the condition were “It has not reduced tumor burden but it has stabilized me and I’m very happy with being stable”, “I like more definitive, not how I feel, but what the tests show. Does the CT scan show that I’m losing lung tissue at a normal rate?” and “It’s the numbers that are going to tell me whether or not this thing is helping, hurting, or not having any effect at all”. Patients with more symptomatic conditions, such as respiratory disease, considered reduction in symptoms to be an indicator of effectiveness. An example quotation was “I think the positive point was getting pain relief”.

Patients with life-threatening conditions such as cancer found immense emotional relief when the medication had a positive physical effect. An example quotation for the impact of effectiveness on emotional/physical well-being was “And in terms of the emotional end…that I’m feeling these different physical things and seeing these different physical results and knowing that I made the extra effort to try and do one more thing for this body to fight cancer is just incredible”.

3.3.2 Burden of Side Effects

Twenty-five patients (78%) talked about the burden of side effects. It was considered the second-most-important concept category for patients with cardiovascular or metabolic conditions but was regarded as low in importance for individuals with cancer (Fig. 2).

Patients expressed their willingness to tolerate some side effects but noted that they would reconsider a treatment if these became a larger factor. Severity, including duration, was considered the most important characteristic when evaluating a side effect, and could determine whether a patient would continue a treatment. Likewise, the extent to which side effects interfered with daily life was viewed as important when assessing the treatment experience. An example quotation for the interference of side effects, from a patient with cancer, was “The bowel changes could be disruptive to daily life, but never in such a way that it made me question the value of the treatment”.

3.3.3 Convenience of Use

Nineteen patients (59%) talked about the convenience of use of a treatment. Generally, convenience was considered to be of lower importance relative to most of the other concept categories and it appeared to relate to how a therapy was administered (Fig. 2). For patients whose treatment required infusion or injection, preparation and storage of the medication was a relevant factor determining convenience of use. An example quotation for preparation/storage of medication was “It’s not convenient to take an injection. I’ve got gel packs that keep everything cold. I can’t go anywhere or travel anywhere”.

When appraising the ease of adhering to and managing the treatment regimen, patients considered the frequency, duration and method of administration of a medication to be important. An example quotation for ease of compliance/managing the regimen was “Is it once a day or is it four times a day? It’s nearly impossible to remember to take a pill three or four times a day. Frequency is important”. Patients did not think the duration of the entire regimen had a major impact on their satisfaction with treatment.

3.3.4 Overall Acceptance/Satisfaction

Eighteen patients (56%) spoke about overall acceptance of and satisfaction with treatment. They mentioned whether they found the treatment acceptable, given their experience with it, but used terms such as “worth taking” rather than explicitly using the word ‘acceptable’. Patients had a range of views on whether effectiveness or lack of side effects was more important when considering overall treatment satisfaction. An example quote, from a patient with thyroid cancer, was “It’s also something that has little side effects…And that is a much better option than taking a pill that causes you to be stable but also takes away your quality of life”. An example quote from a patient with type 2 diabetes mellitus was: “If the drug helps, great. As long as it doesn’t hurt, even better”.

Patients were more likely to be satisfied with a treatment if they believed it had worked, for example from seeing positive effects or experiencing no negative changes. Patient overall satisfaction also depended on whether a reasonable alternative medication was available. An example quotation for impact of unmet need was: “There aren’t any good options for my type of cancer…”.

3.3.5 Cost

Fourteen patients (44%) spoke about cost. Cost was considered to be of lower importance than other concept categories (Fig. 2). However, several patients mentioned that failure to cover the cost of medication by health insurance would be a hurdle to obtaining or remaining on certain treatments. Patients cited the cost of traveling to participate in a clinical trial as burdensome but not a major impediment.

3.3.6 Trial Experience

Sixteen patients (50%) talked about their trial experience. The clinical trial experience was considered relatively important in relation to other concept categories for patients with conditions of low to moderate severity, but less so for those with severe disease states (Fig. 2). Patients talked about both inconveniences and benefits of the trial experience. One of the most commonly cited inconveniences was the amount of time spent on the logistics of participating in the trial, such as time spent traveling to and from the treatment center and time spent at the center. Administrative challenges and experiences with clinical trial staff also contributed to the clinical trial-specific experience. Some patients considered access to free or reduced-cost healthcare to be an important clinical trial benefit.

3.3.7 Altruism

Sixteen patients (50%) spoke about altruism. Altruism was ranked among the three most important concept categories across all primary medical conditions (Fig. 2). Patients considered their contribution to medical research—which they viewed as helping other people with the same health condition—to be one of the most important benefits of participating in a clinical trial. An example quotation for helping others was “What can I do to help? If it’ll help me, great; if it’ll help others, even better”.

3.4 Cognitive Debriefing

Draft items for the benefit–risk instrument were created based on the medication-specific concepts identified during the concept elicitation interviews and were tested during cognitive interviews. Overall, 42 interviews across waves 2–6 included a cognitive debriefing component. The intent was for the final PRO instrument to comprise four items—one each on effectiveness, side effects and convenience of use, to be scored separately, and a final item on overall evaluation, to provide a single metric of the patient’s overall benefit–risk assessment.

3.4.1 Clinical Trial Experience Question

To enable patients to reflect on non-medication-related aspects and to help them separate their perspectives of the trial from those of the medication itself, they were asked upfront in the draft instrument about their clinical trial experience. A specific instruction was added asking patients not to consider their medication experience when responding to this question.

3.4.2 Medication-Related Questions

Instructions were added ahead of the medication-related questions, reminding patients to consider only the study medication in their answers and, if the treatment involved more than one medication, to consider the study medications overall. Participants correctly interpreted the instructions during subsequent testing and no further changes were made.

Patients were able to respond to the question on effectiveness even if they were asymptomatic, for example based on test results. The initial wording of the question (‘How much do you feel the medication helped make your condition better?’) evolved to improve clarity based on patient input during the initial cognitive debriefing interview waves. Other changes were to amend ‘medication’ to ‘study medication’ and to change ‘helped make your condition better’ to ‘help your condition’ to simplify wording, especially as disease stabilization was also considered a benefit by patients. Participants correctly interpreted the question during subsequent testing and no further changes were made.

The original wording of the question on side effects (‘How bad were the side effects of this medication?’) evolved across the interview waves to improve clarity based on patient input. Patients reported that not all side effects (e.g. weight loss) had to be negative, and use of the term ‘bad’, rather than ‘severe’, clarified that this question referred to negative side effects. Not all patients experienced negative side effects; ‘Did not experience any side effects’ was thus added as a response option. An initial gating question with a ‘yes/no’ response option (e.g. ‘Did you have side effects?’) was considered, with patients responding ‘yes’ answering a follow-up question on how bad the side effects had been. However, there was concern that individuals with only ‘slightly bad’ side effects would be unsure how to respond to a ‘yes/no’ gateway question, and it was therefore considered more appropriate for patients to see all available response options without gating. Three patients thought that the response options ‘Not bad at all’ and ‘Did not experience any side effects’ were the same, but other patients understood the options as intended. Participants understood the final side effects question as intended and no further changes were made.

The initial wording of the question on convenience (‘How inconvenient was it for you to be on this medication?’) also evolved based on patient feedback. ‘How inconvenient’ was changed to ‘How convenient or inconvenient’ because patients found ‘inconvenient’ too assumptive; examples of what to consider were added because one patient conflated ‘convenience’ with ‘trial experience’. Subsequent testing showed that no further changes were required.

The benefit–risk question evolved based on patient input during the cognitive debriefing interviews. The prior questions on effectiveness, side effects and convenience of use helped to prepare patients to answer the overall benefit–risk question and provided information on how these different aspects contributed to the overall trade-off response. Draft questions were removed and new questions added after the first cognitive debriefing interview wave. A second item, ‘If you knew before taking this study medication what you know now, how likely would you be to take it?’, was removed because patients also considered aspects of the trial experience such as altruism. The question ‘How would you compare the positive and negative effects of the study medication?’ was understood by patients; however, to avoid a narrow interpretation of the question as referring to effectiveness only, ‘effects of’ was amended to ‘things about’. Response options evolved based on patient feedback.

Alternative rating scales were considered and tested. Patients preferred 5- or 6-point Likert scales with specific response options to numerical rating scales. The instrument will be licensed by AstraZeneca and will be made available for use upon request.

4 Discussion

Understanding how patients perceive the benefit–risk of new medications is crucial and is relevant at all stages of drug design, development and healthcare use. Patients are interested in understanding the benefit–risk of new medications, to know if the ‘good’ outweighs the ‘bad’ when considering a treatment. Understanding patient views of benefit–risk of a new medicine is also important for regulators, who are recognizing that patient-centered benefit–risk assessments need to be incorporated into the clinical development process [3, 5]. In accordance with the US Twenty First Century Cures Act, the FDA is required to evaluate patient experience information as part of the licensing application review [20]. The EMA and the FDA are collaborating on sharing best practices to improve patient involvement along a medicine’s regulatory life cycle, from drug development and evaluation to post-marketing authorization and monitoring activities [3]. PRO data are also considered part of clinical benefit assessments to obtain market access. For payers and physicians, gaining an understanding of the patient’s viewpoint can guide decision making that is in the best interest of those receiving the treatment. Knowing about patient views of benefit–risk of a new medicine is also important for the pharmaceutical industry, which relies on the willingness of patients to use and adhere to the new treatment once marketing approval is obtained.

In this study, we developed a brief PRO instrument, the Patient Global Impression of Benefit–Risk (PGI-BR). The PGI-BR can be used across disease areas to assess the patient’s view of benefit–risk of a new medicine in the clinical trial setting. The instrument aims to measure whether patients in a clinical trial think that ‘good’ experiences outweigh ‘bad’ experiences with the study medication. It consists of four study medication-related questions, one each on efficacy, side effects, convenience of use and overall benefit–risk. Scoring will be further explored. The intent is for each of the four items to be scored separately, with the fourth item providing a single metric of a patient’s overall benefit–risk assessment. Items will be scored on a 5- or 6-point verbal rating scale, ranging from ‘not at all’ to ‘very much’ for efficacy, ‘extremely bad’ to ‘did not experience’ for side effects, ‘very inconvenient’ to ‘very convenient’ for convenience, and ‘negatives far greater than positives’ to ‘positives far greater than negatives’ for overall benefit–risk. The concept elicitation interviews showed that patients considered effectiveness of the treatment, burden of side effects, convenience of use and overall acceptance of and satisfaction with the treatment when weighing the benefits against the risks of a new medicine. Concept saturation occurred early on during the interview process, indicating that all aspects relevant to the target patient populations were captured and supporting the instrument’s comprehensiveness. Concept saturation is also an indicator of adequate sample size in qualitative research [7]. The most appropriate timing of completion of the questionnaire, whether it should take place at the end of a clinical trial or at multiple points throughout, is being piloted, and is particularly relevant in oncology where patients may exit a study at different times. Psychometric work is underway to evaluate psychometric robustness with data prospectively collected in clinical trials. Assessments will include response distributions (including identification of any floor or ceiling effects, if present), factor analysis, item-to-item correlations, internal consistency, test–retest reliability and convergent and known-groups validity. To confirm that the instrument is appropriate for use across a range of conditions, it will be assessed in multiple indications and treatments with varying side-effect profiles and administration schedules.

Understanding how patients view the benefit–risk of a treatment can clarify how willing an individual may be to tolerate negative aspects for benefit; for example, how important ease of use (e.g. oral vs intravenous administration or one vs four times daily dosing schedules) is when weighed against effectiveness. Existing measures such as the TSQM [13, 14], the ‘Was it worth it’ (WIWI) questionnaire [21] and the Patient’s Quality Assessment of Treatment (PQAT) have tried to capture this information with a variety of items, and the FACT-G GP5 (‘I am bothered by the side effects of treatment’) aims to assess the impact of treatment toxicity with a single item [22]. The TSQM was developed to capture patients’ overall satisfaction or dissatisfaction with treatment in relation to its effectiveness, side effects and ease of use, to enable comparisons to be made across medication types and diseases [23]. The WIWI questionnaire endeavors towards assessing patients’ decision-making regarding treatment in a clinical trial [21]. The PQAT was designed to assess the benefits and disadvantages of new treatments for type 2 diabetes mellitus, and was recently used as part of a clinical study [24]. It comprises a question with fixed response options on patients’ overall treatment experience and open-ended questions for free-text responses. In contrast with the current questionnaire, the content of the PQAT was developed without direct patient input, and only underwent cognitive testing in seven patients with diabetes mellitus [24]. By comparison, the current instrument is a patient-driven, patient-centric PRO questionnaire with established content validity using best practices for instrument development [6,7,8,9], and assesses the benefit–risk of ‘good’ and ‘bad’ experiences with an individual treatment across a range of therapeutic areas.

In the current study, it became apparent during the interviews that patients considered non-medication aspects of the clinical trial when they answered questions relating to treatment. To separate out the non-medication aspects, we included an additional question upfront, asking about these aspects of the clinical trial experience. This question provides patients with the opportunity to ‘offload’ their views on non-medication-related aspects of the trial, such as cost benefits, experiences with clinical staff and thoughts around being altruistic. Further guidance was provided with the inclusion of an instruction ahead of the non-medication-related question, asking patients not to consider medication experience in their response. Conversely, ahead of the medication-related questions that make up the main part of the instrument, patients were reminded via an instruction to consider only the medication experience in their responses.

Clinical benefit–risk assessments based largely on clinical study efficacy and safety data are required as part of regulatory submissions for new treatments [2, 4]. To assist with communication of this clinical benefit–risk assessment between the pharmaceutical industry and regulators, a structured framework was developed by the Benefit–Risk Action Team (BRAT) of the Pharmaceutical Research and Manufacturers of America [25, 26]. There are six steps in applying the framework: defining the decision context, identifying the outcomes, identifying and extracting the source data, customizing the framework, assessing the importance of the outcome to decision makers or stakeholders and displaying and interpreting key benefit–risk metrics [25, 26]. The BRAT framework is a best-practice industry tool for assessing benefit–risk. It is our hope that a patient-centric measure such as the PGI-BR will in future be incorporated into the framework to ensure that the patient viewpoint is taken into account.

This work has several key strengths. Development of the PGI-BR was based on patient input early on, using qualitative concept elicitation and cognitive interviews, thus providing strong content validity. Patients with a variety of primary medical conditions (oncological, respiratory, metabolic and cardiovascular) were included in the interviews. The final instrument is brief and does not demand much respondent time. The relatively small number of patients in each disease area, in particular cardiovascular conditions, is a potential limitation. Although diversity was sought with respect to some aspects, further use of the instrument is needed to provide insight into diverse populations by ethnicity. As with other instruments used in clinical trials, the perspectives are of those in a trial setting, and patients in the real world might have different experiences and views.

5 Conclusions

We have developed a brief PRO instrument, the PGI-BR, that can be applied across disease areas to assess the patient view of benefit–risk of a new medicine in the clinical trial setting. Regulators and payers in particular make critical decisions based on their judgment of overall benefit–risk trade-offs that determine whether patients have access to treatments. Patients who use the treatment and who ultimately experience its benefits and risks should have a voice in regulatory and reimbursement decisions, and the information should be available to them. The PGI-BR is currently being assessed in clinical trials and a scoring mechanism is being developed to provide an interpretable benefit–risk metric. How the instrument is scored, and interpretation of those scores, is pending analysis of quantitative data. Once this development and validation work is complete, the PGI-BR will be made available for use.