1 Introduction

There are many approaches to assessing the patient’s perspective on medical treatment [1, 2]. Among these approaches, patient preference studies have taken on an increasingly important role in assessing the benefits and risks of medical devices. Patient preference studies can provide a number of transparent and defensible measures of the patient perspective on treatment benefits, risks, and other characteristics, including an identification of what features of medical device treatments matter to patients, quantitative estimates of how much each feature matters, the tradeoffs patients are willing to make among the treatment features, and an estimate of the proportion of patients who would perceive the benefits of a medical technology to outweigh its risks [3, 4]. Therefore, patient preference information can inform medical device development strategy, development planning, regulatory submission, and post-approval assessment.

Patient engagement has become a top strategic priority for the US Food and Drug Administration (FDA)’s Center for Devices and Radiological Health (CDRH). Recognizing the value of the patient perspective in medical device development in particular, the agency committed to increasing the use and transparency of patient input as evidence in its decision-making. The agency’s guidance documents [5, 6], provisions within the “Medical Device User Fee Amendments 2017 (MDUFA IV)” commitment letter [7], CDRH’s Patient Preference Initiative [8, 9], and its 2016–2017 Strategic Priorities [10] all demonstrate this commitment. While the focus of this article is on medical devices in the USA, there are also numerous regulatory, government, and legislative initiatives in drugs worldwide that focus on patient input [11,12,13,14,15,16,17,18,19].

2 Patient Preferences in the FDA Regulatory Review of Medical Devices

To understand how patient preferences might be used to inform benefit–risk assessments of medical devices, the Medical Device Innovation Consortium (MDIC), a public–private partnership among the FDA, the medical device industry, research organizations, and nonprofits, proposed a “Framework for Incorporating Information on Patient Preferences Regarding Benefit and Risk into Regulatory Assessments of New Medical Technology” [3].

In addition, there are a number of recent examples of using patient preferences to inform regulatory benefit–risk assessments. CDRH conducted its own proof-of-concept study of patient preferences for the benefits and risks of weight-loss devices and approved a device, in part, on the basis of those preference data.

CDRH’s obesity study was the first such study carried out by a regulatory agency. Published in 2015, the study by Ho et al. quantified benefit–risk preferences for weight-loss devices from among a sample of overweight and obese people in the USA using a discrete-choice experiment [4]. The authors used the results of the study to develop a tool to calculate (1) the minimum amount of weight loss a patient would require in order to accept the risks associated with a novel weight-loss device, (2) the maximum mortality risk patients would tolerate in exchange for a given amount of weight loss, and (3) the proportion of patients in the sample who likely would perceive the benefits to outweigh the risks of alternative weight-loss device profiles.

CDRH used the tool to apply patient preferences to the clinical data of a novel weight-loss device being considered for marketing authorization in the USA [20]. The decision to approve the device was the first such approval based, in part, on patient preference information. According to CDRH, preference data indicated that a group of patients were willing to accept the risks of the surgically implanted device in exchange for the weight-loss benefits. The patient preference data, along with long-term data showing sustained weight loss, led to the approval decision [20].

Other recent examples of the FDA using patient preferences to inform regulatory decisions are a recent FDA advisory committee approval of a subcutaneous formulation of rituximab in the treatment of blood cancers, supported in part by a patient preference study conducted within the trial [21, 22], and the release of an FDA draft guidance [23] on developing treatments for Duchenne muscular dystrophy, in part informed by a proposed draft guidance developed by the patient group Parent Project Muscular Dystrophy, which conducted a patient preference study to support their draft [24].

Most recently, CDRH has committed to advancing patient input and patient involvement in the regulatory process as part of the “Medical Device User Fee Amendments 2017 (MDUFA IV)” commitment letter [7]. CDRH will use industry user fees to conduct additional preference studies and develop additional expertise and capacity to respond to submissions containing patient preference studies, including early consultation and advice to industry during preference study planning. The agency also plans to hold a series of public meetings by fiscal year 2020, whose topics would center around advancing the science of patient input, patient engagement, developing preference study examples and case studies, and “identifying priority areas where decisions are preference sensitive and patient preference data can inform regulatory decision-making” [7].

The CDRH weight-loss study and CDRH’s patient-engagement commitments demonstrate progress on the journey to routinely incorporating patient preference into regulatory decision-making. The CDRH guidance on incorporating patient preference information in regulatory submissions identifies multiple factors that industry sponsors should consider when developing a patient preference study [6]. However, researchers have noted that there remain many key questions to answer before decision makers can be confident in using patient-preference information to inform regulatory benefit–risk assessments [25]. In addition, MDIC identified several areas in which additional work may be necessary to understand the properties of patient preference methods and the implications of these properties for the use of patient preference information for regulatory purposes [3].

While the application of preference studies to issues in healthcare is not new, the use of these studies in a regulatory context is relatively new and raises a new set of questions for researchers, industry, patient groups, and regulators to consider. Mainly, what should the regulatory requirements be for these types of studies? Furthermore, to what degree would the current approaches for conducting the studies meet those requirements?

The purpose of this editorial is to describe these research needs and to suggest a set of empirical studies, the results of which will begin to address these outstanding questions. We view this as a call to action for researchers to address these gaps and advance the science of patient preferences for use in regulatory assessments of the benefits and risks of medical devices.

3 Defining and Understanding the Unknowns Associated with Patient Preference Studies

The increasing application of preference studies to healthcare and the recent endorsement of such studies by CDRH has raised a new set of questions around regulatory requirements and the ability of current studies and methodologies to meet these requirements [3]. To answer these questions, we must first understand the unknown factors in play. While this list is by no means exhaustive, we believe that the unknowns listed below provide a starting point for near-term research.

3.1 Unknown: Validity and Reproducibility of Existing Methods

How do we ensure a patient preference study is valid?

Validity, as a concept, is well defined in other areas of health research, such as patient-reported outcomes; however, no similar consensus on the meaning(s) of validity exists in patient preference research. Some preference methods, particularly those such as discrete-choice experiments which ask repeated questions of patients, allow researchers to assess the consistency of responses within a single study. Similar tests are not possible for other methods, such as the threshold technique and multi-criteria decision analysis. In addition, there are no established standards for evaluating the consistency of results obtained by conducting the same study with different methods or populations.

3.2 Unknown: Effect of Method Choice on Study Results

When answering the same research question, do different methods or parameterizations of these methods lead to the same conclusions?

There is very limited evidence comparing the results of applying different methods to measure patient preferences for benefits and risks for use in regulatory decision-making. This is especially the case when comparing methods that come from different research disciplines, such as economics, decision science, psychology, market research, and health services research. We are aware of some recent and ongoing work in this area [26,27,28]; however, further research and potentially a theoretical approach are necessary.

3.3 Unknown: Effect of Sample Composition on Study Results

Which patients’ preferences matter and how do we best define the sample? How sensitive are patient preference results to the choice of sample?

The CDRH patient preference guidance dictates that a patient preference study “measure the preferences of a representative sample of adequate size so that the study results can be reasonably generalized to the population of interest” [6]. A key task is defining to what that representativeness refers, for example, patient clinical characteristics, patient demographic characteristics, or patient preferences. Sample diversity is also a key consideration if we want to understand subgroups and the effect of prior experience on preferences. Additionally, there is the somewhat philosophical question of how the choice of which patients to study (e.g., those at risk for the disease, those newly diagnosed, those recently treated, or those experienced with a chronic disease) impacts the results and their application.

3.4 Unknown: Effect of Attribute Inclusion or Exclusion on Study Results

How do we properly determine which attributes should be included and which should be excluded from the study?

What qualitative approaches can best identify these attributes and how they should be described in a survey instrument, when qualitative research is necessary to address the research question?

How sensitive are the results to the choice of attributes?

How would results differ if two surveys had equally valid but different phrasing for the same set of attributes?

Preference survey results depend completely on the treatments and treatment attributes included in the study, i.e., the benefits, harms, and other features of treatment that form the basis of the survey. The research question dictates the attributes selected for the study. But uncertainty exists as to how sensitive the results are to the inclusion or exclusion of particular attributes. Because the selection of attributes and their definitions may influence results, we must better understand the extent of that influence.

3.5 Unknown: Effect of Communication Style and Methods on Study Results

What is the most appropriate format and level of information to provide when asking patients to make preference decisions?

Patients’ life experiences, existing knowledge, and the information presented to them during the study will influence their responses to preference questions. How do we take prior experience and knowledge into account and how much weight do we then assign to those data? Furthermore, how do we ensure that patients are not making decisions blindly, but also that they are not bombarded with unnecessary information? When can this information induce bias in the results? No definitive standard for the appropriate level of information or its presentation exists.

4 Bridging the Gap: Future Areas of Research to Advance the Science of Patient Preference Assessment

For each of the unknowns posed in the previous section, we suggest a set of empirical studies, the results of which will begin to address these outstanding questions.

4.1 Future: Validity and Reproducibility of Existing Methods

Consider three lines of research in parallel: (1) Conduct a literature review of standards and methods for establishing validity across other fields, such as patient-reported outcomes. Evaluate these standards to understand how they are implemented in other areas of research to provide a basis for establishing analogous standards for validity in preference research. (2) Convene a series of workshops with preference experts to address these questions from a theoretical, analytical, and empirical basis. (3) Conduct theoretical, analytical, and simulation research work to provide evidence to support proposed standards for validity.

4.2 Future: Effect of Method Choice on Study Results

Use multiple patient preference methods, especially those which arise from different research disciplines or that take a very different theoretical approach to preference assessment, to answer a common research question to determine the extent to which different methods support the same conclusion. Compare method performance and consistency, highlight advantages and limitations of each methodological approach, identify similarities and differences in the types of preference information provided by each method, and determine the implications of these differences for informing regulatory benefit–risk assessments.

4.3 Future: Effect of Sample Composition on Study Results

Conduct studies comparing the preferences of patients in different populations, such as those with and without prior experiences with disease outcomes, those with and without prior treatment experience or with different treatment experiences, or those recruited through different mechanisms (e.g., clinical studies, clinical centers, or internet panels). Conduct a longitudinal study to assess the effect of disease and treatment experience on patient preferences over time.

4.4 Future: Effect of Attribute Inclusion or Exclusion on Study Results

Use different qualitative and quantitative approaches to identify and define attributes to elicit preferences for the same technology. In a multiple-arm study, use the same preference elicitation method with each set of attributes and attribute definitions to understand whether the inclusion or exclusion of attributes results in comparable estimates of relative importance or tradeoffs.

4.5 Future: Effect of Communication Style and Methods on Study Results

Create different definitions for the same attributes and include questions to test patients’ comprehension of the definitions. Develop alternative decision contexts to elicit preferences within a survey. In a multiple-arm study, evaluate the degree to which attribute descriptions, attribute comprehension, and decision context affect the resulting estimates of relative importance or tradeoffs.

5 Moving Forward

Patient preference information is one type of evidence that can be used to apply the patient perspective to clinical safety, efficacy, and tolerability in regulatory benefit–risk assessments. Although some examples of the application of patient preference data to regulatory decisions exist [4, 22, 29], many questions remain about how to incorporate patient preference data in regulatory decisions and the standards that should be applied to these data.

A number of organizations, including, but not limited to, patient groups, such as the National Health Council and FasterCures, industry associations, such as the Pharmaceutical Research and Manufacturers of America and the Biotechnology Innovation Organization, and public–private partnerships, such as the Medical Device Innovation Consortium in the USA and the Innovative Medicines Initiative in Europe, have suggested or are evaluating frameworks for systematically incorporating patient preference information in regulatory benefit–risk assessments. However, our purpose here is to recommend concrete, empirical studies to advance existing knowledge of preference assessment. In doing so, we hope to help make it possible for industry, patient groups, and regulators to have confidence in the quality of the results and ensure that they are rigorous and free from bias.

The door is open for research to advance the science of patient preference assessment to inform medical device regulatory benefit–risk decisions so that it becomes commonplace in FDA review for the benefit of patients worldwide.