FormalPara Key Points for Decision Makers

Despite growing recognition of the potential benefits of patient and public involvement (PPI) and the formal requirement by many funders to include PPI in research, we found a limited number of preference studies that utilised PPI activities.

The case study demonstrates that PPI activities can be beneficial across the stages of preference research.

Using the case study and published literature, we highlight that there are many potential stages for PPI activities in preference research that will achieve a potentially impactful and less tokenistic level of engagement.

1 Background

Production of research and its subsequent translation into practice can be a challenging process. A key facilitator to ensuring wider relevance of research is the involvement of stakeholders from the start of the research, and this approach is increasingly promoted across different areas of healthcare in the production of healthcare [1] and by funders of research [2]. In healthcare, the involvement of public stakeholders is commonly referred to as ‘Patient and Public Involvement and/or Engagement’ (PPI or PPIE) [3]. The encouragement of PPI is often based on the principle that patients and the public are key consumers of healthcare and therefore benefit from healthcare research [3, 4]. In creating a partnership between researchers and the public or patients, it is anticipated that research becomes more relevant to the potential beneficiaries [5]. Throughout the paper, when we discuss PPI we refer to the INVOLVE definition as “public involvement in research as research being carried out ‘with’ or ‘by’ members of the public rather than ‘to’, ‘about’ or ‘for’ them” [6].

Health economics is a broad discipline concerned with efficiency, effectiveness, behaviour and equity. As such, there are many ‘non-research’ stakeholders to consider; for example, patients, caregivers, clinicians and policy makers in local and national government institutes or agencies all of whom may have an interest in the development or results of research questions. A key fundamental of health economics is to maximise benefits (often health) to a certain population usually under constrained resources [7]. Consequently, much research conducted by health economists may have an impact in the ‘real world’. For example, research may affect how hospitals are organised or inform decisions by the National Institute for Health and Care Excellence that may affect patients’ access to medicines [8, 9]. To maximise benefits in markets that are imperfect, economists may use stated preference methods to quantify the value placed on goods or services [10]. Discrete choice experiments (DCEs) have been applied in health since the 1990s [11]. In a DCE, individuals select their preferred option from a series of hypothetical goods or services, which are described in terms of fixed attributes but vary in their levels [12]. The choices made reveal the trade-offs they are willing to make and therefore the relative importance of the attributes in their decision making. Recent reviews have shown DCEs to be used in a wide range of applications from labour market choices to health-state valuations [11, 13].

The results of DCE research are having an increasing impact on healthcare practice and decision making. At the ‘micro-level’, preference data are informing shared decision-making tools to help patients and clinicians make informed choices [14,15,16]. At the ‘macro-level’, regulatory decisions regarding product safety are being made on behalf of whole populations based on benefit-risk evidence produced by DCE studies [17,18,19]. For example, the US Food and Drug Administration is developing guidance for quantitative preference elicitation studies [20] and the European Medicines Agency is involved with the Innovative Medicines Initiative PREFER project, which aims to increase the role of quantitative patient preference data throughout the drug life cycle [21]. Results from preference studies can also be submitted alongside other types of evidence as part of a health technology assessment [22]. Understanding preferences through the use of DCE research can also help to align healthcare interventions to preferences [15].

There are now many best practice guidelines for conducting DCEs [23, 24] including different aspects of design [25] and analysis [26]. There are also guidelines specifically for conducting qualitative research to develop preference instruments [27]. However, there are no specific guidelines about how to conduct effective PPI throughout the preference study process. Note that typically, patients informing the design of experiments fall under the heading of qualitative research, for example, as part of focus groups [28] or interviews [29]. Patient and public involvement specifically refers to members of the public carrying out research with or without the support of other researchers, which is a two-way process (if researchers are involved), compared to qualitative research in which the researchers are informed by participants [30]. The literature notes that there remains uncertainty in how to incorporate PPI into healthcare research, which may lead to suboptimal impact and a feeling that engagement was purely tokenistic [31].

We first present examples from the existing published literature, describing the key stages of involvement outlined by the authors and any benefits or challenges discussed. We then consider in more detail a DCE case study taken from a recent National Institute for Health Research-funded research programme aimed at improving psychological outcomes in cardiac rehabilitation (CR), which demonstrates that PPI can be used at several stages throughout the design, creation, recruitment and reporting of preference research. In addition, we detail the benefits and challenges of including PPI in the case study research. Finally, drawing together the existing literature, the case study and wider PPI literature, we describe how PPI activities may provide insights at each stage of the preference study process to encourage researchers to consider PPI at all stages of the research, moving away from minimal tokenistic engagement.

2 Examples from Stated Preference Literature

Examples of PPI activity informing preference research were identified from the published literature by searching for keywords, such as “patient involvement”, “public involvement”, “stakeholder involvement” and “stakeholder engagement”, in studies identified by a previous systematic review of DCEs (searches updated to capture evidence published up to January 2018) [11]. Note that full details on the methods and results for the review can be identified in the referenced published paper. Over 300 health-related DCEs have been published since 2013, however, we only identified a handful of examples that described the stakeholder engagement (including patient representatives and groups) in the article or report. An overview of these studies, with a focus on engagement activities, is provided in Table 1. The identified studies did not necessarily refer to PPI activities, rather patients and the public (including caregivers) were among key groups included in wider stakeholder engagement activities and alternative terminology was often used.

Table 1 Overview of examples from the published literature
Full size table

The identified examples did not typically provide a thorough overview of the benefits and challenges of engagement; however, this is understandable given word limits on journal articles and that the main focus of the authors would be communicating key methods and study results. The extent of engagement across stages of preference research varied, most commonly it was used to develop attributes, although there are examples of it being used at other stages (e.g. improving the readability of survey materials and interpreting results).

The papers noted some limitations of the stakeholder engagement; including challenges interpreting attribute terminology, missing attributes, the time taken for the process, not all members attended every meeting and potential bias in the sample/representativeness of the sample [34, 35, 37]. Though it was noted in the papers, missing attributes is not necessarily specific to stakeholder engagement, as it is possible that this may occur if stakeholder engagement is not used to assist in identifying attributes and attribute development also needs to consider feasibility (limits on the number of attributes). Furthermore, using stakeholder engagement alongside other methods (e.g. literature review and qualitative research) may reduce the likelihood of missing attributes by considering a wider perspective. All identified studies were positive about stakeholder engagement, despite limitations. Peay discussed that although the design of such an experiment is complex, it can be led by an advocacy organisation (with expert collaboration) and noted that the community engagement was particularly advantageous in achieving recruitment targets [37]. dosReis et al. commented that stakeholder advisors were well informed and knowledgeable, and subsequently their inclusion as coinvestigators added a depth of understanding to enhance the research [34]. Wittenberg [39] provides a useful commentary on the benefits of stakeholder engagement in stated preference research, focusing on the study by Janssen et al. [35].

3 Case Study

The literature review presents some useful examples demonstrating potential stages of involvement throughout the process of preference research and an overview of potential limitations and benefits. However, the literature presented limited detail on involvement activities. The case study below reports PPI across the stages of a DCE aiming to provide a more thorough overview of the aspects of involvement and the associated benefits and limitations.

3.1 Study Objectives

The PATHWAY Programme is a 6.5-year project funded under the UK National Institute for Health Research Programme Grants for Applied Research (RP-PG-1211-20011) [40]. The study aims to improve access to more effective psychological interventions for patients attending CR who present with symptoms of depression and/or anxiety. As part of the PATHWAY programme of work, a multicentre, two-arm, single-blind, randomised controlled trial is being conducted to compare the clinical and cost effectiveness of group-based metacognitive therapy plus CR with CR alone. In addition to the trial, a DCE is being conducted to estimate preferences for psychological care within the CR pathway. This case study focuses on the PPI activities related to the DCE work specifically. The DCE research aims to determine the relative importance of characteristics of the psychological intervention in CR, and to assess how people trade between these characteristics. This information can help services target improvements in CR to the aspects that are most important to current and future potential participants of CR.

3.2 Establishing a Working Partnership

The PPI group (the PATHWAY Patient and Public Advisory Group) was formed to provide support, guidance and advice at all stages of the research (not just the DCE activities), and to ensure that the study aims, procedures and dissemination were informed by those with lived experience. To be eligible as a member of the PPI group, potential members had to have experience of one of the following: heart disease, anxiety and/or depression, or being a carer of someone with one or more of these conditions. The group were originally recruited via patient networks, such as Salford Citizen Scientist, the Ticker Club and Salford Heart Care (advertisement included in the Electronic Supplementary Material [ESM]). Initially, there were ten PPI members, which included a chair member who was involved in the grant application; however, two members have left, and another has stepped back, the reason typically given for withdrawing from the PPI group was ill health. Of the seven current members of the PPI team, the majority are female and over the age of 65 years (further details included in the ESM). During the PPI activities related to the DCE, the majority were male, which is aligned with the general population undergoing CR [41]. The PPI group is coordinated and supported by the PPI lead, with meetings held two or three times per year. Patient and public involvement across the whole project will be reported in a separate paper.

3.3 Stages of Patient and Public Involvement Activity

An overview of the PPI activity in the preference work is presented in Fig. 1 and took place over five sessions conducted with the PPI group. The stages of PPI activity were determined through researcher experience and discussion with PPI coordinators. The research team was keen to include PPI throughout the process.

Fig. 1
figure 1

Stages of the PATHWAY patient and public involvement

Full size image

As a first step, the group was given introductory training focused on health economics and stated preference survey techniques. Members were introduced to the reasons for conducting and the benefits of preference research. To illustrate choice sets, two examples from the literature were used; one looking at preferences for health states and a second focused on preferences for primary care consultations [42, 43]. Following training, the group was asked to focus on the research question, which they agreed was clear. However, the PPI group recommended that the term metacognitive therapy was replaced with psychological therapy, as it would be a more familiar term and therefore prevent confusion. This was agreed by the research team as results are likely to be relevant to psychological therapies more widely.

The biggest area of PPI activity was step 3 (developing the choice questions); the first stage of this work involved splitting into two smaller groups to generate initial ideas on what members might consider or want to know prior to attending therapy (e.g. location/setting, information provided prior to starting therapy and time required), with refinement in later sessions to reduce the number of attributes and levels down. Prompts for potential attributes were provided based on expert opinion, the design of psychological therapy, qualitative interview feedback and a review of the literature. In the case study, a separate qualitative component of work was conducted; however, the primary aim was not to inform the DCE design, rather to explore the impact of cardiac events on patients and their psychological needs [44, 45]. However, qualitative interview feedback was reviewed and supported the DCE design by providing some initial ideas for attributes. When discussing key attributes, there were some instances where the PPI group disagreed on the importance of attributes, and a majority consensus was reached. Note that the PPI group found it clearer if the researchers referred to “characteristics” rather than “attributes” and therefore this language was used in the PPI activities and survey development. A summary of step 3 and step 4 (demographics) is provided in the ESM.

Once survey materials were drafted, following review and revisions by clinical and academic experts, PPI members completed them independently to review and provide feedback on contents and clarity. This also helped to inform the estimate for the time taken to complete the survey. One useful idea from the PPI group was that reading from white paper can be challenging for people with dyslexia and offering coloured paper copies of surveys may be beneficial. The majority of participants in the planned survey are likely to be identified via the PATHWAY trial and a market research company; however, in the case of issues with recruitment (e.g. slow recruitment or a sample not representative of the population), the PPI members were asked to provide a list of recommended patient groups to target to ensure recruitment targets are met.

Finally, as part of the wider PPI work, the group was asked to co-produce the dissemination plan for the PATHWAY trial. The group identified key messages, target audiences, formats, publications/venues and suggested approaches to develop a series of ‘off-the-shelf’ dissemination ‘products’ for a public audience. These suggestions informed draft products that the group then modified through group discussion and e-mail. This process can be started in advance of study results allowing time for discussion around the selection and translation of results before the end of the project. It is worth noting that while the group ultimately decided to focus on the development of public facing dissemination, involving them in the initial idea generation for all target audiences can be helpful in generating novel ideas.

3.4 Challenges and Limitations

The debate between PPI members during group activities demonstrated the complexities of choices and showed that participant characteristics will likely impact choices in a real-world setting. Managing disagreements in a PPI group can be challenging when members have different views and when some members of the group are more vocal than others. However, by splitting the group into two (approximately four members in each), it was kept manageable and everyone had an opportunity to feedback their opinion. In addition, a further session was needed to refine the list of characteristics of therapy (attributes for inclusion in the DCE design) as initially it was an unmanageable size. Note, these challenges may be more prominent when the groups are diverse or larger. It is recommended that preference researchers ensure a PPI co-ordinator is present at meetings as they provide vital assistance when dealing with any challenges and facilitate communication between researchers and PPI members. A further challenge was the length of time between meetings and unavoidable absences, especially from the first meeting. This resulted in the need to recap the training on preference research. While an effective solution, this did have an impact on the time available for activities within the meeting. As with all engagement work, the views of PPI participants may not be representative of the wider population. Though the group was typical of the population undergoing CR (typically retired and a higher proportion of male individuals) during DCE-related activities, it is important to consider that feedback will not reflect the total CR population [41]. Additionally, because members have withdrawn owing to ill health, the structure of the group has changed (the majority are now female).

3.5 Concluding Thoughts

It was anticipated that the PPI group might find the training and activities around the preference research uninteresting; however, this was not the case. When given the opportunity to feedback on their involvement in the wider PATHWAY project, the PPI members remarked that they felt that one of their areas of greatest impact had been related to the preference work. In contrast, with using a purely qualitative approach to identifying attributes and levels for a DCE, the inclusion of PPI (seeing the PPI as research collaborators, rather than research subjects as they would be in qualitative research) had benefits. In particular, it was an iterative process that allowed researchers to obtain feedback at several stages of design and the PPI group had an introduction to preference research and felt suitably informed and involved. Though it may be seen as less systematic than a qualitative research study with a more rigorous protocol, there were benefits to the less formal approach, such as having the ability to go back to the PPI team to reduce the number of attributes and levels. Overall, activities with the group were successful; PPI members drew on their experience to identify a range of characteristics that people attending CR may consider before deciding on whether to attend metacognitive therapy sessions and to identify the participant characteristics likely to have an impact on choices. It was noted by the wider PATHWAY team not only to be useful for the DCE design, but for wider practice and policy, especially when considering the low uptake rates associated with CR [41].

4 Fostering Patient and Public Involvement in Stated Preference Research

A stated preference study has multiple inter-related steps from developing the choice tasks to constructing the experimental design and collecting data from respondents for statistical analyses. In this section, we describe how PPI activities may provide insights at each stage of the preference study process, informed by the examples from the literature, the case study and discussion between authors (preference researchers and a PPI coordinator). This is not to say that PPI must be included across all stages, rather to provide researchers with an overview of how PPI can be useful across the process of preference research and to encourage researchers to move away from including PPI at a single stage, which can seem tokenistic. Furthermore, we recognise that the usefulness of PPI will vary according to the research question, methods (e.g. experiment design) and existing literature.

The case study highlighted the importance of introductory training about health economics and preference research to the PPI group, as well as recap sessions if the design process is lengthy. It is recommended that future studies consider optional recap sessions prior to activities. An example of how health economics can be communicated to PPI researchers is included in the book “A Research Handbook” [46].

4.1 Defining the Research Question

With any research project, the first stage is often developing and defining the research question [23]. In some instances, this may be defined by a particular policy but in many instances specifying the study perspective and rationale for research may be difficult, particularly if the results of the research have relevance for multiple groups, e.g. patients and their caregivers. A review of PPI in health and social care research identified that it can help to contribute towards user-focused research objectives and user-relevant research questions [47]. In addition, it is important for the research question to be clear and understandable to potential participants, PPI can help to ensure participants clearly understand the aim of the research they are contributing to, as demonstrated in the PATHWAY case study.

4.2 Selecting Attributes and Levels

There is some consensus that attributes and levels should be identified using qualitative research methods with individuals that represent the sample for the final stated preference survey [27]. Using PPI activities to identify attributes and levels may have parallels to using qualitative research methods in this context. For example, researchers collaborating with stakeholders should take a rigorous approach, documenting or recording changes to the research pathway and why these occurred, as well as reflecting on their own biases and how these may influence participants’ contributions. However, there may be some advantages to using stakeholder engagement in addition to purely qualitative approaches [34, 35, 37, 38]. For example, advisory boards may provide a useful opportunity to elicit other practical insights regarding implementation that may not emerge in qualitative research. dosReis et al. [34] engaged stakeholders in the analysis of qualitative data collected to identify attributes and levels. Engaging stakeholders in this capacity may provide an opportunity to pragmatically reduce many themes emerging in expansionary qualitative research to a reasonable and plausible (not just desirable) set for use in a choice experiment. Seeing PPI as a two-way communication could be a more iterative process that allows for greater discussion and debate around design (vs a one-way qualitative method of obtaining public feedback). Note that we are not suggesting that PPI replaces qualitative methods, rather that it can add to several stages of preference research.

4.3 Choice Set Design

Stakeholder engagement may assist at the early stages of the study to understand the most appropriate choice format (e.g. contingent valuation, DCE, best–worst scaling). For example, Coxon et al. chose a ‘partial profile’ design for their DCE study after the research user group expressed that six attributes were too cognitively burdensome [33]. Another key consideration for many researchers is whether to have a forced-choice experiment. To relax the assumption positive demand (i.e. a respondent will always choose to consume one of the alternatives presented), choice experiments may include an ‘opt-out’ or ‘status quo’ option. However, there is evidence to suggest that individuals can bring different meaning to the ‘none’ alternative [48]. Stakeholder engagement exercises may be useful in revealing the specification (levels, terminology) for an opt-out alternative.

4.4 Design of Survey Materials

Qualitative interviews to formally pre-test the survey instrument are likely to be seen by many health preference researchers as the ‘gold standard’ of design. However, an initial survey needs to be developed for use in any pilot work. Patient and public involvement activities may be an ideal opportunity to refine language and terminology to encourage respondents to correctly and consistently interpret descriptions and questions. Seo et al. [38] used two committees comprising experts and community stakeholders to ensure descriptions were appropriate to patient respondents completing the survey but would also resonate with the medical community. In the PATHWAY case study, members of the PPI group were given copies of the draft stated preference survey materials and made some interesting recommendations. Some were presentation related, such as printing on coloured paper to help people with visual impairments and bolding text with the closing date, others focused on simplifying the wording and some were more practical, e.g. telling people when to expect reimbursement. This also allowed the team to receive initial feedback on how long the survey may take. A review of PPI in randomised controlled trials described that one of the most reported benefits was related to writing participant information sheets, such as improving the clarity of information [5], another review explained that it can help to develop user-friendly information, questionnaires and interview schedules [47].

4.5 Data Collection

A key challenge in preference research is recruiting an adequate sample [49]. Stakeholder engagement groups may be able to provide practical advice and personal insights into different recruitment strategies. For example, community leaders have been used to create and distribute newsletter notices and recruitment e-mails and raise awareness through word-of-mouth recruiting in health preference research [37]. Patient and public involvement activities have been linked to improved recruitment materials [50], more appropriate recruitment strategies for studies [47] and an increased likelihood of recruiting the necessary sample within restricted timeframes [50,51,52,53]. In addition, stakeholder groups may be able to advise on approaches to recruit lesser reached groups or provide advice on appropriate introductions or reimbursement. In a previous stated preference study by one of the authors, the involvement of a service user researcher was imperative to obtaining a sufficient sample size of secondary care users of mental health services, to assess individual preferences for key aspects of care planning [54]. This included the service user researchers making use of their network, contacts and known organisations to boost recruitment. In addition, in the PATHWAY case study, PPI members were keen to ensure that the sample reached would include older people who do not regularly use social media or who may not see posters in clinics and therefore recommended attending patient groups in the community, such as The Ticker Club. It must be noted that PPI advice around design and recruitment must be considered alongside the timeframe and budget for the study.

4.6 Interpreting the Results

Involving the PPI group in the results stage of preference research can have several benefits. The PPI members can check that the results are clear and understandable to a lay audience. In addition, the group can help to interpret results, especially anything that seems unexpected. Adams et al. explored a surprising finding that some parents preferred having no financial incentives for vaccinating children. The parent advisory group suggested that this could be seen as a reward for ‘bad behaviour’ for those who had not vaccinated their child prior to the introduction of a financial incentive [32]. In the previously discussed example with secondary care users of mental health services (assessing individual preferences for key aspects of care planning), there was one unusual result,in which participants preferred a level “agree” to “completely agree” for risk management [54]. It had been assumed that there would be a linear trend for preferences (i.e. completely agree would be favoured over agree). However, the service user researcher was able to help interpret as they were aware that participants may be wary of the negative connotations of risk and placing too much emphasis on this aspect,therefore, whilst service users would still want to see risk management considered, they would not want it to be the focus. This is something that the research team would have been unlikely to explain without the assistance of a service user. The PATHWAY case study has not yet reached this stage, however, during the design of attributes and levels, the PPI feedback was recorded and will be used to support the interpretation of results.

4.7 Generating Impact

There are a number of ways that PPI members can be involved in the dissemination phase; from summarising the findings in an accessible way to supporting as co-authors on an academic paper. In addition, they can provide ideas that help the research to reach a wider and more diverse audience (such as patient groups or forums). Morgan et al. describe how the results of a poster presentation of their results was shown to the mother-and-baby group [36]. The group offered suggestions about how the research could be disseminated including key websites, leaflets and newsletters. The literature reviewed in this article also recognises that the involvement of PPI groups in dissemination may result in more innovative dissemination activities, which can help to increase impact, such as social media blogs [55]. The PATHWAY PPI members have generated ideas for dissemination. The literature notes that PPI in dissemination is most likely to impact on implementation and changes to practice [56].

5 Conclusions

Patient and public involvement and related activities provide opportunities to collaborate with key stakeholders to guide the development of research and ensure the outputs are relevant to those who may be affected. Acting on opportunities to engage patients and/or the public in a collaborative sense rather than using these individuals as research subjects may yield different but equally interesting insights. This paper outlines potential stages of PPI activity across the design, recruitment and dissemination processes related to preference research to support researchers and to encourage PPI in this work. Note that the paper focuses on DCE research as the case study used is a DCE, however, we expect that the findings are relevant and applicable to researchers involved in a broad range of preference research, particularly those using hypothetical survey-based approaches.

There is the need to avoid a sense of tokenistic engagement, which has been associated with funders requiring PPI [5, 52]. This may be in part owing to a feeling of uncertainty around how and when to involve PPI [31]. We would encourage all researchers to ensure that PPI activity will be used to inform the research before patient and public contributors are contacted. Based on the published literature, it appears PPI has been rarely reported in DCE studies despite PPI being a recommended or required at all stages of research by funders [57]. This suggests that PPI is not always included as part of preference research, or that authors feel it did not make a substantial enough contribution to the research to warrant reporting (i.e. it was perceived as a “tick box” exercise to appease funders). Additionally, the identified examples use engagement activities at a limited number of stages across the research. We outline several stages in which PPI can contribute to preference research, which may help researchers to create a more comprehensive and less tokenistic sense of engagement, where the PPI group are clearly contributors/collaborators to research.

The existing literature discusses the many possible benefits from PPI activities, including clearer survey materials, timely recruitment and innovative dissemination. In addition, we feel that specific to preference research, PPI members may be able to offer practical insights and pragmatic approaches to refining themes for inclusion in preference studies and furthermore, can make valuable observations that support researchers interpreting results. We anticipate that the usefulness and opportunities for PPI activities will vary depending on the precise study research questions and study design. However, researchers will not always be aware of the potential benefits of PPI in the planning stages. For example, in Sect. 4.6, there are two examples presented where it helped to interpret unexpected results from preference research [54, 58]. Although we encourage researchers to think carefully about the incorporation of PPI and to ensure that PPI activities inform the research (i.e. the PPI groups time is not wasted on meaningless activities), the full benefits will not always be known until the end of the research. Researchers must consider the feasibility (e.g. time and cost restrictions) and the potential benefits of PPI when planning a study.

There are challenges associated with PPI activity, typically focused on time and funding requirements [5, 52]. Wider recent reviews have noted inconsistencies in the reporting of PPI activities and that researchers report difficulties in providing information on PPI activities, which perhaps can partially explain the lack of examples identified [5, 59]. New checklists, such as the GRIPP2, can support and encourage researchers to more comprehensively report PPI activities in preference research [60]. Inadequate management of expectations has been noted as a potential barrier to successful PPI and we recommend that researchers and PPI members outline expectations prior to conducting the work, to ensure a shared vision of the research [61]. Recommendations for managing expectations are outlined in the wider PPI literature [51]. Further challenges include difficulties translating complex terminology and teaching groups adequately to allow them to contribute more thoroughly [61], low meeting attendance or PPI members dropping out (illustrated in the case study) [61]. Finally, caution needs to be taken as groups may not always be representable of the wider population and members may act in their own interests or provide biased feedback [62].

There are some limitations to this work, most notably, recommendations based on author opinion/reflection have been drawn using a single case study and examples from the published literature. As PPI is encouraged by organisations and funders, it is likely that there will be a growth in the relevant literature in the future that may strengthen processes. Therefore, we recommend researchers keep up to date and ensure they learn from new examples to ensure they use PPI activities most effectively. To identify examples from the literature, the authors used results from a previous systematic review and searched the included publications for key terms related to PPI [11]. The search was updated to identify more recent results, but this was not systematic, and the terms used to identify examples (e.g. “patient involvement”, “stakeholder engagement”) were kept broad to account for differences in terminology but may not have captured every term used to describe PPI. Therefore, it is possible that the review did not identify every example from the literature. However, as the purpose was to identify some existing published examples and to summarise these, this is unlikely to affect the conclusions of this paper. A full systematic review of PPI contributions to preference research would incur several challenges around the differences in terminology. Additionally, it is outside the scope of this paper. This paper focuses on how PPI can inform DCE research. It does not consider how best to carry out PPI activities, for example, choosing PPI members, effective facilitation and dealing with challenges. We recommend that DCE researchers work closely with an experienced PPI facilitator and review the wider PPI literature related to effective practices. In addition, the paper focuses on a UK setting because of the use of the case study conducted in England and author experience. However, we think that it is possible for researchers across all countries to include PPI at multiple stages of preference studies and think the ideas presented will be generalisable to researchers in other countries. Preferred terminology and methods related to PPI activities and preference research more widely are likely to vary according to country and/or funders. Researchers will need to consider what is appropriate in the context of their own study and setting. The case study presented benefitted from having funders who encourage PPI and a PPI coordinator to support work. The feasibility, relevance and potential for benefit related to PPI activities is likely to differ across specific projects and requires careful consideration from researchers. Finally, the role of this paper is to support researchers planning PPI in preference research. As the use of PPI in preference research expands, further research systematically comparing methods related to PPI in preference research and how they impact on the agreed methods and results of studies would strengthen the literature.

As discussed, the examples presented demonstrated that PPI can be very useful in preference research. Initial concerns that the case study PPI members might find the DCE research uninteresting were unfounded. It is also worth remembering that there are benefits for PPI members, for example, a project in cancer research discussed that participants valued the opportunity they had to take part in research and the knowledge and skills they acquired throughout the process [63]. The PATHWAY PPI members have also noted that they benefitted from being part of the group as it provided an opportunity to talk to others who had been through similar experiences, they learnt new information and they felt they “had a voice”. Therefore, researchers should be encouraged, knowing that PPI members will also benefit from robust research processes.