Skip to main content

Methodological Priorities for Patient Preferences Research: Stakeholder Input to the PREFER Public–Private Project

Patient advocacy groups, regulatory agencies, and industry have increasingly advocated for patient engagement in decisions across the medical product lifecycle (MPLC). Among the array of approaches to obtain patient input, an increasingly popular and important approach is the use of patient preferences, including an understanding of which endpoints are most important to patients and of the patient perspective on benefit–risk (B-R) trade-offs when making treatment and reimbursement decisions [1,2,3,4]. This push to involve patient preference information throughout the MPLC has resulted in a growing body of knowledge and experience in this field [5,6,7,8,9,10,11], which in turn has stimulated a growing interest in how best to conduct patient preference studies [4, 12,13,14,15,16,17,18]. Despite the increasing frequency with which patient preference studies are conducted, there remain many unanswered questions regarding how to incorporate scientifically valid preference measurements into MPLC decision making regarding medical treatments, including development, regulatory and reimbursement decisions. Previous groups such as the Medical Device Innovation Consortium (MDIC) have worked to address these issues but many questions remain [15, 19]. To answer some of these questions, the European public-private partnership PREFER (‘Patient Preferences in Benefit and Risk Assessments during the Drug Lifecycle’) was launched in 2016 [20].

PREFER is a 5-year project, funded jointly by the Innovative Medicines Initiative (IMI) 2 (EU Horizon 2020) and the European pharmaceutical industry [represented by the European Federation of Pharmaceutical Industries and Associations (EFPIA)]. PREFER aims to strengthen patient-centric decision making throughout the MPLC by developing evidence-based recommendations to fill the gaps in knowledge regarding the methodological aspects of patient preference studies.

The early stages of PREFER focused on assessment of the patient preference landscape to outline research needs. This assessment was based on systematic literature reviews and both individual interviews and focus groups with key stakeholders (i.e. academic researchers, physicians, regulators, health technology assessment/payer representatives, industry representatives, patients, caregivers and patient representatives). The literature reviews focused on the methodological aspects of patient preference assessment methods and previously conducted studies [21,22,23,24,25]. The interviews and focus groups were conducted with more than 140 stakeholders from seven European countries and the United States. These were used to identify the desires, expectations, concerns, and requirements of stakeholders about methodologies for patient-preference elicitation and their use in making well-informed decisions regarding medicinal products [22, 26,27,28,29]. Based on the findings of this early work within PREFER, a multistep approach was used to draft a research agenda for PREFER partners and other parties interested in patient preference information.

The first step in drafting the research agenda involved using the results of the literature reviews and stakeholder interviews to develop over 100 questions on the methodology, design, conduct, and application of preference studies. Irrespective of the point in the MPLC, the most important research needs identified were related to four high-level concepts: evidentiary standards, assessment of preference heterogeneity, means to minimize patient burden, and means to maximize patient understanding of concepts presented in preference studies. These four high-level concepts were consistent across the literature, methods assessments, interviews and focus groups. The second step involved refining, clustering and categorizing the questions into three tiers based on their priority and suitability to be examined in prospective case studies within the PREFER project and in subsequent preference studies conducted after the PREFER project has concluded (Fig. 1). The criteria for categorization for each tier were as follows:

  • Primary tier: Questions (n = 27) relate to the validity and reliability of preference methods, including consistency across preference studies using different methods, adjustments in attributes, and/or different samples. These questions (1) can be examined in a patient preference case study; (2) focus on more promising preference methods [28]; and (3) had not been well-studied as of March 2018, according to PREFER partners, stakeholders and external scientific advisors.

  • Secondary tier: Questions (n = 40) that (1) can be appropriately addressed in a case study but already have some evidentiary basis, either from previous preference studies or from psychometric research or related disciplines; or (2) relate to topics that are relevant to conducting a preference study, such as planning, organization and set-up.

  • Tertiary tier: Questions (n = 39) that (1) cannot be appropriately addressed in a case study; or (2) are related to the use and interpretation of patient preference study outcomes.

Fig. 1
figure 1

Process to determine the PREFER research agenda and question prioritization for case studies. *One question was classified under two themes depending on the context of the question

The classification of research questions into tiers was agreed to by the PREFER consortium and scientific advisory board at the PREFER Annual Meeting in October 2018. After establishing this general research agenda, partners in the PREFER consortium were asked to rank their top five questions in the primary tier according to their priority for being addressed in a prospective case study conducted as part of PREFER.

In total, 33 members of PREFER partners responded to the survey, resulting in the identification of 17 prioritized questions, as shown in Table 1. The top-tier questions relate to three themes: the reliability and validity of preference outcomes, the generalizability and transferability of results, and the impact of educational materials. The reliability and validity questions were generally ranked as the highest priority. These questions involve comparison of different types of preference elicitation methods, modulation of specific aspects within a method, or assessment of similar methods across different samples of patients drawn from the same patient population. An example of this type of question is “How do results differ between simpler/cheaper methods versus more complex/expensive methods?

Table 1 Prioritized questions by prioritization rank and question theme

Questions related to the generalizability and transferability of patient preference study findings were ranked as a second highest priority. These questions cover topics related to understanding aspects that may explain preference heterogeneity, such as differences in recruitment channels, patient characteristics, including psychosocial constructs such as health literacy and numeracy or locus of control, and variation in preferences across stakeholder groups. This theme was especially highly prioritized by stakeholders who work in clinical settings or directly in patient care. An example of this type of question is “Can measures of psychosocial constructs serve as covariates that are predictive of preference for particular diseases?

Finally, questions related to educational tools used to inform participants in patient preference studies, and which patient factors to measure to best understand patient preference study outcomes were ranked third most important. An example of these questions is “How do results differ when participants are presented with information in a scenario-based interactive tool versus traditional text-based education?

Based on the final rankings of research questions, each prospective PREFER case study team was encouraged to develop research questions related to these three themes. To this end, the PREFER case studies were asked to include at least one prioritized reliability question, to assess health numeracy and literacy measurements along with other psychosocial constructs, and to address other prioritized questions where possible given the patient population, disease context, and potential design of the case study.

The findings of the PREFER project regarding which questions should be prioritized for preference research were developed through expert consensus. These findings corroborated and extended the patient preference research agenda developed by the MDIC, whose patient preference report was published the year prior to the launch of PREFER [15, 19]. The PREFER project operationalized these agendas by producing a specific set of higher-priority questions to be used to guide the design of prospective case studies. The next steps for PREFER include examining these higher-priority research questions across case studies in three prespecified patient groups (neuromuscular diseases, lung cancer, and rheumatoid arthritis), nine additional academic and industry-led case studies across a wide array of disease and treatment contexts, and simulation studies during the 5-year project.

The questions generated within PREFER cannot all be answered in a fixed number of case studies and the relatively limited timeframe of the PREFER project. Rather, PREFER will advance the field by focusing on topics that require a large public–private collaboration to initiate, and lay the groundwork for future researchers to add to and improve our understanding of patient preferences. To this end, PREFER researchers encourage others to utilize the specific research questions presented in this article to conduct future studies, build off the methodological insights from the PREFER case studies, and contribute toward answering the highly prioritized questions. The collective knowledge created by the collaboration will result in a strong body of evidence to help increase the understanding and utilization of patient preference studies across the MPLC.


  1. Soekhai V, De Bekker-Grob EW, Ellis AR, Vass CM. Discrete choice experiments in health economics: past, present and future. Pharmacoeconomics. 2019;37(2):201–26.

    Article  PubMed  Google Scholar 

  2. Bridges JFP, Kinter ET, Kidane L, Heinzen RR, McCormick C. Things are looking up since we started listening to patients. Patient Patient-Cent Outcomes Res. 2008;1(4):273–82.

    Article  Google Scholar 

  3. Brett Hauber A, Fairchild AO, Reed JF. Quantifying benefit-risk preferences for medical interventions: an overview of a growing empirical literature. Appl Health Econ Health Policy. 2013;11(4):319–29.

    CAS  Article  PubMed  Google Scholar 

  4. Marsh K, Van Til JA, Molsen-David E, Juhnke C, Hawken N, Oehrlein EM, et al. Health preference research in Europe: a review of its use in marketing authorization, reimbursement, and pricing decisions—report of the ISPOR stated preference research special interest group. Value Health. 2020;23(7):831–41.

    Article  PubMed  Google Scholar 

  5. Hoos A, Anderson J, Boutin M, Dewulf L, Geissler J, Johnston G, et al. Partnering with patients in the development and lifecycle of medicines. Ther Innov Regul Sci. 2015;49(6):929–39.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Mavris M, Furia Helms A, Bere N. Engaging patients in medicines regulation: a tale of two agencies. Nat Rev Drug Discovery. 2019;18(12):885–6.

    CAS  Article  PubMed  Google Scholar 

  7. Haerry D, Landgraf C, Warner K, Hunter A, Klingmann I, May M, et al. EUPATI and patients in medicines research and development: guidance for patient involvement in regulatory processes. Front Med. 2018;5:230.

    Article  Google Scholar 

  8. Anderson M, Kimberly MK. On the path to a science of patient input. Sci Transl Med. 2016;8(336):336ps11-ps11.

    Article  Google Scholar 

  9. European Medicines Agency. ICH reflection paper on proposed ICH guideline work to advance patient focused drug Development. EMA/CHMP/ICH/415588/2020. Committee for Medicinal Products for Human Use, European Medicines Agency; 2020.

  10. European Medicines Agency. The patient's voice in the evaluation of medicines: how patients can contribute to assessment of benefit and risk. EMA/607864/2013. Stakeholders and Communication Division, European Medicines Agency; 2013.

  11. US FDA. Patient-focused drug development: collecting comprehensive and representative input; guidance for industry, food and drug administration staff, and other stakeholders. Silver Spring: Center for Drug Evaluation and Research, US FDA; 2020.

    Google Scholar 

  12. US FDA. Patient Preference Information – Voluntary Submission, Review in Premarket Approval Applications, Humanitarian Device Exemption Applications, and De Novo Requests, and Inclusion in Decision Summaries and Device Labeling: Guidance for Industry, Food and Drug Administration Staff, and Other Stakeholders. Silver Spring: US FDA; 2016.

  13. Ho M, Saha A, McCleary KK, Levitan B, Christopher S, Zandlo K, et al. A framework for incorporating patient preferences regarding benefits and risks into regulatory assessment of medical technologies. Value Health. 2016;19(6):746–50.

    Article  PubMed  Google Scholar 

  14. Reynolds RF, Kurz X, De Groot MCH, Schlienger RG, Grimaldi-Bensouda L, Tcherny-Lessenot S, et al. The IMI PROTECT project: purpose, organizational structure, and procedures. Pharmacoepidemiol Drug Saf. 2016;25(Suppl 1):5–10.

    Article  PubMed  Google Scholar 

  15. Medical Device Innovation Consortium (MDIC). Patient centered benefit‐risk project report: a framework for incorporating information on patient preferences regarding benefit and risk into regulatory assessments of new medical technology. Medical Device Innovation Consortium; 2015.

  16. Postmus D, Richard S, Bere N, Van Valkenhoef G, Galinsky J, Low E, et al. Individual trade-offs between possible benefits and risks of cancer treatments: results from a stated preference study with patients with multiple myeloma. Oncologist. 2018;23(1):44–51.

    Article  PubMed  Google Scholar 

  17. Mühlbacher AC, Juhnke C, Beyer AR, Garner S. Patient-focused benefit-risk analysis to inform regulatory decisions: the European union perspective. Value Health. 2016;19(6):734–40.

    Article  PubMed  Google Scholar 

  18. Johnson FR, Zhou M. Patient preferences in regulatory benefit-risk assessments: a US perspective. Value Health. 2016;19(6):741–5.

    Article  PubMed  Google Scholar 

  19. Levitan B, Hauber AB, Damiano MG, Jaffe R, Christopher S. The ball is in your court: agenda for research to advance the science of patient preferences in the regulatory review of medical devices in the United States. Patient Patient-Cent Outcomes Res. 2017;10(5):531–6.

    Article  Google Scholar 

  20. De Bekker-Grob EW, Berlin C, Levitan B, Raza K, Christoforidi K, Cleemput I, et al. Giving patients’ preferences a voice in medical treatment life cycle: the PREFER public-private project. Patient Patient-Cent Outcomes Res. 2017;10(3):263–6.

    Article  Google Scholar 

  21. Russo S, Jongerius C, Faccio F, Pizzoli SFM, Pinto CA, Veldwijk J, et al. Understanding patients’ preferences: a systematic review of psychological instruments used in patients’ preference and decision studies. Value Health. 2019;22(4):491–501.

    Article  PubMed  Google Scholar 

  22. Van Overbeeke E, Whichello C, Janssens R, Veldwijk J, Cleemput I, Simoens S, et al. Factors and situations influencing the value of patient preference studies along the medical product lifecycle: a literature review. Drug Discov Today. 2019;24(1):57–68.

    Article  PubMed  Google Scholar 

  23. Janssens R, Huys I, Van Overbeeke E, Whichello C, Harding S, Kübler J, et al. Opportunities and challenges for the inclusion of patient preferences in the medical product life cycle: a systematic review. BMC Med Inform Decis Mak. 2019;19(1):189.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Soekhai V, Whichello C, Levitan B, Veldwijk J, Pinto CA, Donkers B, et al. Methods for exploring and eliciting patient preferences in the medical product lifecycle: a literature review. Drug Discovery Today. 2019;24(7):1324–31.

    Article  PubMed  Google Scholar 

  25. Whichello C, Bywall KS, Mauer J, Stephen W, Cleemput I, Pinto CA, et al. An overview of critical decision-points in the medical product lifecycle: where to include patient preference information in the decision-making process? Health Policy. 2020;124(12):1325–32.

    Article  PubMed  Google Scholar 

  26. Van Overbeeke E, Janssens R, Whichello C, Schölin Bywall K, Sharpe J, Nikolenko N, et al. Design, conduct, and use of patient preference studies in the medical product life cycle: a multi-method study. Front Pharmacol. 2019;10:1395.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Janssens R, Russo S, Van Overbeeke E, Whichello C, Harding S, Kübler J, et al. Patient Preferences in the medical product life cycle: what do stakeholders think? Semi-structured qualitative interviews in Europe and the USA. Patient Patient-Cent Outcomes Res. 2019;12(5):513–26.

    Article  Google Scholar 

  28. Whichello C, Levitan B, Juhaeri J, Patadia V, Disantostefano R, Pinto CA, et al. Appraising patient preference methods for decision-making in the medical product lifecycle: an empirical comparison. BMC Med Inform Decis Mak. 2020;20(1):114.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Van Overbeeke E, Forrester V, Simoens S, Huys I. Use of patient preferences in health technology assessment: perspectives of Canadian, Belgian and German HTA representatives. Patient Patient-Cent Outcomes Res. 2021;14(1):119–28.

    Article  Google Scholar 

Download references


The authors would like to thank members of the PREFER consortium for their contributions in informing the research agenda. The research priorities expressed herein were determined by discussions and ranking exercises among the PREFER members. This text and its contents reflect the PREFER project’s view and that of the authors, but not the view of the IMI, EU, EFPIA, or the authors’ respective organization(s).

Author information



Corresponding author

Correspondence to Ian P. Smith.

Ethics declarations


The PREFER project has received funding from the IMI 2 Joint Undertaking under grant agreement number 115966. This joint undertaking receives support from the EU’s Horizon 2020 Research and Innovation Program and the EFPIA.

Conflicts of interest/Competing interests

Conny Berlin is employed by, owns stock in, and has stock options in Novartis Pharma AG, which is one of the industry partners in the PREFER Project. Rachael DiSantostefano and Bennett Levitan are employed by Janssen Research and Development, LLC, and are stockholders in Johnson & Johnson. Bennett Levitan also owns stock in a variety of companies that at times include other pharmaceutical and health care-related companies. Ian P. Smith, Esther W. de Bekker-Grob, Jorien Veldwijk, and G. Ardine de Wit declare they have no conflicts of interests that are relevant to the contents of this article.

Availability of data and material

The datasets generated and/or analysed during the current study are available from the corresponding author on reasonable request.

Code availability

Not applicable.

Author contributions

IS created the first draft of this commentary in collaboration with RD. All authors were contributors to the PREFER research agenda technical report, on which this commentary is based, and all authors reviewed and provided comments on the draft and approved the final version before submission.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Smith, I.P., DiSantostefano, R.L., de Bekker-Grob, E.W. et al. Methodological Priorities for Patient Preferences Research: Stakeholder Input to the PREFER Public–Private Project. Patient 14, 449–453 (2021).

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: