Introduction

Pancreatic cancer (PC) is the sixth leading cause of cancer death in Australia [1] and the fourth in the United States, [2] as disease outcomes remain dismal, with single digit 5-year survival rates both in Australia [3] and elsewhere [2]. Despite this, research in PC is lagging behind than in other cancers, [4] with only 1% of the research budget of the US National Cancer Institute being spent on PC [5].

In the absence of agreed processes to reliably identify research areas that can deliver significant advances in PC research, the Cancer Council NSW (CCNSW) established a strategic partnership with the NSW Pancreatic Cancer Network to systematically define critical research issues and opportunities that could accelerate progress in this field in Australia.

Here, we describe the priority-setting process, the outputs generated at each stage and how they informed PC research funding decisions.

Materials and methods

The study commenced with a wide consultation with PC researchers, clinicians, and consumer representatives to identify available options for research prioritisation. Consensus development methods were deemed to be the most likely to be able to provide stakeholder agreement on PC research priorities.

The process consisted of five distinct stages: a literature review on recent progress in PC research, semi-structured expert interviews, a Delphi process, consumer focus groups (FGDs), and a nominal group process (NGP). The literature review together with expert interviews was used to provide a “level playing field” for all participants, irrespective of their specific area of expertise, to facilitate their contribution to the Delphi process. We decided to use FGDs to ascertain consumer priorities, in order to maintain an independent consumer voice throughout the study. Information about priorities identified by the experts through the Delphi process and by consumers through FGDs were provided to nominal group participants, to provide them with a comprehensive picture of potential research priorities to consider and prioritise (see Fig. 1). The study was approved by the relevant Institutional Ethics Committees.

Fig. 1
figure 1

Diagrammatic representation of the priority-setting process

Full size image

Literature review

A comprehensive literature review examined the progress in PC research since 2001, when the NCI multidisciplinary Progress Review Group published its analysis of the state of PC research [4].

Expert interviews

Ten international experts took part in semi-structured telephone interviews discussing the current status of PC research. Participants were invited to suggest research directions most likely to provide the next major research breakthrough in PC, to identify barriers limiting progress in research, and to identify where Australian research could make significant contributions (see Table 1).

Table 1 Summary results of semi-structured interviews and experts’ recommendations
Full size table

Delphi process

The Delphi method has been developed “to obtain the most reliable opinion consensus of a group of experts by subjecting them to a series of questionnaires in-depth interspersed with controlled opinion feedback” (p. 458) [6]. Delphi methods commonly involve several rounds of questioning, with de-identified individual responses collated, analysed, and summarised before being returned to respondents in a subsequent round; the process is repeated until ‘consensus’ is achieved [7]. Modifications of the Delphi technique have been used for developing clinical guidelines and quality indicators [7–12], clinical decision aids [13], identifying research priorities [14–19], defining priorities in cancer care [20], and identifying health practitioner’s educational priorities [21].

Delphi participants

The Delphi panel consisted of 27 experts in basic science, epidemiology, clinical medicine, and policy makers, both from Australia and overseas. Participants were provided with the expert group’s recommendations and literature reviews as background information. Five rounds of questioning and opinion feedback were conducted, with each round consisting of collecting de-identified responses to an emailed questionnaire. Participant responses were pooled and analysed by the research group, then circulated again as ‘feedback’ in a subsequent round [6].

Consumer focus groups

The focus group discussions (FGD) sought participants’ views on key research areas for PC, identifying the unmet needs of people with PC and inviting comment on initiatives that would be most helpful to patients and their carers.

We used FGDs to ascertain consumer views about PC research, as the flexible nature of questioning allows the exploration of the wide range of opinions and attitudes held by participants [22]. We could not identify previous reports describing ways to engage with PC patients around research prioritisation, likely related to the difficulty of carrying out research with people who suffer from a highly lethal cancer.

Conducting FGDs by telephone, rather than in the traditional face-to-face format was dictated by limitations in health status and mobility imposed by a PC diagnosis and allowed patients with advanced cancer and their carers to become engaged in the priority-setting process with minimal disruption to their lives. One discussion group comprised patients with a PC diagnosis, the other involved current and bereaved carers of patients with PC. Separating the groups allowed discussions to focus on issues specific to each group and how research could address them.

The aim of the research was outlined in a newsletter of the Pancreatic Cancer Support Project, distributed to people affected by PC. Interested potential participants were invited to register their interest in participating and participants were assured of confidentiality of the information.

The facilitated discussions were digitally recorded and subsequently transcribed. Two experienced telephone support group counsellors facilitated the discussions and an independent qualitative researcher played an observer role.

Nominal group process

The aim of the nominal group was to elicit participants’ views on PC research priorities most likely to accelerate progress and to develop consensus around a final list of Australian PC research goals. Participants included clinicians (medical and radiation oncologists, surgeons, and palliative care physicians), pathologists, basic science researchers, epidemiologists, and nurses.

The process comprised three steps: the identification of new research directions and issues, a discussion about key research directions identified through group work and the refining and tabling of the final research goals.

The second step involved each participant selecting and ranking their top three research priorities, while taking into account their feasibility within the Australian context. This involved consideration of existing Australian research capacity, availability of resources and the potential of the identified priorities to deliver a competitive advantage to Australian researchers. The ranked research priorities were subsequently discussed within the group and grouped into categories.

Results

Literature review

The literature review focussed on five key areas of research activity identified by the NCI Pancreatic Cancer Progress Review Group: [23] (1) disease aetiology and prevention, (2) early detection and screening, (3) PC diagnosis, assessment, and staging, (4) PC treatment and (5) palliative and supportive care.

Expert interviews

The expert group recommended a focus on the following research directions: developing better imaging and screening technologies, identifying genes that can be targeted by therapy, focusing on drug discovery, developing new animal models for PC, encouraging multidisciplinary research collaborations, emphasising epidemiological research and developing well-annotated clinico-pathological databases. Australian researchers were perceived to have a competitive advantage in database development and developing research networks that could advance PC research (see Table 1).

The Delphi process

In Round 1, participants were invited to contribute up to ten issues they believed could change outcomes in PC in the Australian context (n = 22, 73% response rate), with responses collated and circulated to the group in Round 2. As a substantial and very diverse number of research priorities (134) were identified, a group of PC experts devised ‘research categories’ enabling the grouping of similar or related items.

In Round 2a, the list derived from Round 1 and grouped by specific research category was circulated. Participants (n = 19, 63% response rate) were invited to confirm whether the process accurately captured their suggested priorities and invited to add (or edit) the statements to better encapsulate their views. These responses were used to inform Round 2b, where participants (n = 20, 66% response rate) were asked to nominate their five top priority areas from the final list. At that stage, research priorities selected by fewer than 15% of respondents (fewer than two participants) were excluded, resulting in 14 research priorities progressing to Round 3.

In Round 3, the relevance of these research priorities was ranked using a Likert scale, from 1 (research unimportant) to 5 (research very important); 25 respondents took part in this round (83% response rate).

Results of this round were circulated to participants in the fourth and final round, ranked in order of their importance, as identified in Round 3. Six research priorities received an average score >4 points, with the remaining eight priorities averaging between three and four points. The six highest ranking priorities and their average score are listed in Table 2.

Table 2 Top 6 research priorities identified by the Delphi process and their average scores
Full size table

Consumer focus groups

A range of issues were identified by both carers and patients as priorities for PC research. Four main themes were identified, related to the following: PC diagnosis, research into disease causation, better consumer and practitioner information about PC, and the quality of pancreatic cancer care. Different subthemes were identified and the information structured using a spreadsheet format. A summary of issues is presented in Table 3.

Table 3 Priorities identified by consumer focus groups
Full size table

There was concordance between the patient and the carer group as to research priorities in PC, with most recommendations drawn from individual experiences. Specific research recommendations included the following: researching better modalities to deliver chemotherapy, reducing the deleterious effects of pancreatectomy on nutrition, finding simpler and faster screening tests, developing targeted screening for people at increased risk of PC, and exploring the role of genetic testing in disease prevention.

Nominal group process

New research issues and challenges identified by participants included administrative difficulties in the research application process, the impact of an increased use of complementary and alternative medicines, the need to distinguish issues germane to research from health delivery issues, to develop suitable trials for patients with advanced PC, and expediting the process of clinical trial selection for this group. In the final step, the most frequently listed research priorities were merged and refined into seven research goals (see Table 4).

Table 4 Final list of research priorities identified by the nominal group process
Full size table

The nominal group recommended the publication and dissemination of these research goals and suggested that future grant applications should identify specific research challenges and how to address them.

Discussion

This paper summarises a process of research prioritisation for a cancer which so far has eluded the attention of funding bodies, despite the significant disease burden it imposes. We used a combination of methods for defining research priorities, utilising the outputs of two consensus development methods (Delphi and nominal group processes) and focus group discussion to arrive at a list of “actionable” research goals specific to Australian PC research. The reason for combining several consensus building methods was to rule out that findings being influenced by the method chosen, although previous studies comparing the Delphi method to nominal group found that both methods performed equally well for predictive purposes [24]. In our study, research priorities generated through the Delphi process and the NGT were very similar and echoed consumer priorities identified through FGDs, validating our approach. Using a Delphi panel, a modified Delphi panel, and a survey-based panel, Brown et al. [25] identified medical conditions for which the risk of hospitalisation is related to ambulatory care access and found that consensus techniques had good predictive ability in this context.

A retrospective review of pancreatic cancer cases reported to a Cancer Registry in Burgundy (France) made similar recommendations regarding future PC research: focus on a better understanding of genetic and environmental causes of PC, on mechanisms involved in its development and on the role of diet in PC risk identified; key challenges related to detecting early/asymptomatic disease and improvements in therapy to improve outcomes [26].

Given the variety of opinions that may exist when a diverse group considers a highly technical and polarising topic, formal consensus approaches (the Delphi method, the nominal group process, and the consensus development conference) are increasingly used to make complex decision in medicine and health [27]. This is based upon the finding that in situations where the evidence limited, unclear or results diverge widely, consensus methods can provide “a bridge between clinical reasoning and clinical research” [28].

Traditionally cancer research priorities have been determined using a “group” model of decision-making, where options are advocated for by spokespersons and a judgement of quality and merit made by the group. While a wide range of experience and knowledge of group participants presents distinct advantages for group models of work [24], the output of large committees may be subverted by domineering personalities, or because group members may be unwilling to take a position before all facts are known, to change it once it is stated publicly, or to publicly contradict senior participants [29].

Our group chose consensus development methods in preference to committee or expert reference panels, due to their perceived limitations and based on the proven track record of formal consensus methods in achieving research prioritisation in the health arena [16, 17, 19, 30–34]. It was encouraging to note also that four of the seven research priorities identified by the nominal group arose also from the consumer FGDs recommendations.

Discussions in the planning phase of the research led to a decision not to include consumers in the Delphi process, to safeguard against the potential “lay disempowerment” that may arise when bringing together experts and consumers. Qualitative researchers have questioned whether the notion of “empowerment” reflects participants acquiescing researcher’s views, rather than empowerment to express their own views [35]. While studying Australian research priorities in mental health, Griffiths et al. [36] found significant differences in research priorities between the committees responsible for evaluating research grants and consumer and carer groups. We wanted to be able to identify specific consumer and researcher perspectives on research priorities before merging them into a final set of recommendations through the nominal group process.

A telephone-based system can present specific challenges for the conduct of FGDs, as important non-verbal cues may be overlooked in this setting. A key element in obtaining quality data and providing a good experience to participants is to ensure that moderators are experienced in eliciting the views of severely ill patients. In this project, the FGDs were conducted by a pair of specially trained telephone counsellors working with people with cancer. They worked closely with a qualitative researcher with expertise in conducting FGDs, to ensure that participants had a positive experience and that all participants had the opportunity to have their views heard. The FGDs produced a large amount of key information relatively expeditiously and in a way that allowed the participation of consumers who were unwell and would not otherwise have the opportunity to have their voice heard.

Study limitations

While the Delphi process involved a panel of 27 experts, their heavy workloads precluded their participation in every round, so response rates ranged between 63 and 85% per round. However, the inclusion of multiple rounds—including a final opportunity to dispute or add to the group’s consensus—enables us to feel confident that the 14 priorities identified reflect the views of the entire panel.

We appreciate that the findings of the FGDs cannot be used to make statements about what all those affected by PC think about key research priorities, but the level of concordance reached independently by patients and carers would suggest that to some extent at least these results are generalisable to the larger group. While focus groups are intended to encourage participants to talk with each other, rather than answer the moderators’ questions, this presented specific challenges for a telephone mode of delivery. Using experienced support group moderators was essential in ensuring group interaction and to provide an in-depth understanding of consumer’s views.

The FGD recommendations, together with the results of the Delphi process were discussed in a nominal group. This enabled the list of research priorities to be discussed in a face-to-face setting, alleviating some of the criticisms levelled at the Delphi methodology, such as the potential of manipulating the consensus, unreliability of the process, and doubts as to what represents acceptable level of consensus [37, 38].

The nominal group defined and ranked seven research directions for Australian PC research. We critically summarised the findings of the consensus process and fed them back to the participants for comment. The priority list was circulated to researchers in the field, inviting applications for a new funding scheme—the Innovator Grants. The grants were open to all applicants who addressed the research priorities and the application process was simplified, in keeping with suggestions received from nominal group participants. Twenty-three applications were received and six were funded through the new scheme, closing the loop from research prioritisation to implementation.

Conclusion

This prioritisation exercise provided a much needed “road map” for research prioritisation in pancreatic cancer, providing valuable information to researchers applying for PC research grants. Additionally, the consumer FGDs provided insights into what patients and their carers see as critical for PC research: better and earlier diagnostic modalities, improvements in understanding disease aetiology, high quality information provision about all aspects of the disease and its treatment and devising better ways in communication between doctors and patients. It was encouraging to note that many consumer research priorities rose to the top of the priority list and will hopefully provide new research insights in PC.