Abstract
Introduction
Pancreatic cancer (PC) is the sixth leading cause of cancer death in Australia and the fourth in the United States, yet research in PC is lagging behind that in other cancers associated with a high disease burden. In the absence of agreed processes to reliably identify research areas which can deliver significant advances in PC research, the Cancer Council NSW established a strategic partnership with the NSW Pancreatic Cancer Network to define critical research issues and opportunities that could accelerate progress in this field in Australia.
Materials and methods
The process consisted of five distinct stages: a literature review on recent progress in PC research, semi-structured expert interviews, a Delphi process, consumer focus groups, and a nominal group process. Information collected at each step informed the development of subsequent stages.
Results
The results from these steps were refined by the nominal group into a set of seven specific pancreatic cancer research goals. The goals were disseminated and led to a new funding scheme for key PC research priorities.
Discussion
This prioritisation exercise provided a much needed “road map” for research prioritisation in PC and served as a checklist to researchers applying for PC research grants to confirm how their research can contribute towards accelerating progress in PC research in Australia.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Pancreatic cancer (PC) is the sixth leading cause of cancer death in Australia [1] and the fourth in the United States, [2] as disease outcomes remain dismal, with single digit 5-year survival rates both in Australia [3] and elsewhere [2]. Despite this, research in PC is lagging behind than in other cancers, [4] with only 1% of the research budget of the US National Cancer Institute being spent on PC [5].
In the absence of agreed processes to reliably identify research areas that can deliver significant advances in PC research, the Cancer Council NSW (CCNSW) established a strategic partnership with the NSW Pancreatic Cancer Network to systematically define critical research issues and opportunities that could accelerate progress in this field in Australia.
Here, we describe the priority-setting process, the outputs generated at each stage and how they informed PC research funding decisions.
Materials and methods
The study commenced with a wide consultation with PC researchers, clinicians, and consumer representatives to identify available options for research prioritisation. Consensus development methods were deemed to be the most likely to be able to provide stakeholder agreement on PC research priorities.
The process consisted of five distinct stages: a literature review on recent progress in PC research, semi-structured expert interviews, a Delphi process, consumer focus groups (FGDs), and a nominal group process (NGP). The literature review together with expert interviews was used to provide a “level playing field” for all participants, irrespective of their specific area of expertise, to facilitate their contribution to the Delphi process. We decided to use FGDs to ascertain consumer priorities, in order to maintain an independent consumer voice throughout the study. Information about priorities identified by the experts through the Delphi process and by consumers through FGDs were provided to nominal group participants, to provide them with a comprehensive picture of potential research priorities to consider and prioritise (see Fig. 1). The study was approved by the relevant Institutional Ethics Committees.
Literature review
A comprehensive literature review examined the progress in PC research since 2001, when the NCI multidisciplinary Progress Review Group published its analysis of the state of PC research [4].
Expert interviews
Ten international experts took part in semi-structured telephone interviews discussing the current status of PC research. Participants were invited to suggest research directions most likely to provide the next major research breakthrough in PC, to identify barriers limiting progress in research, and to identify where Australian research could make significant contributions (see Table 1).
Delphi process
The Delphi method has been developed “to obtain the most reliable opinion consensus of a group of experts by subjecting them to a series of questionnaires in-depth interspersed with controlled opinion feedback” (p. 458) [6]. Delphi methods commonly involve several rounds of questioning, with de-identified individual responses collated, analysed, and summarised before being returned to respondents in a subsequent round; the process is repeated until ‘consensus’ is achieved [7]. Modifications of the Delphi technique have been used for developing clinical guidelines and quality indicators [7–12], clinical decision aids [13], identifying research priorities [14–19], defining priorities in cancer care [20], and identifying health practitioner’s educational priorities [21].
Delphi participants
The Delphi panel consisted of 27 experts in basic science, epidemiology, clinical medicine, and policy makers, both from Australia and overseas. Participants were provided with the expert group’s recommendations and literature reviews as background information. Five rounds of questioning and opinion feedback were conducted, with each round consisting of collecting de-identified responses to an emailed questionnaire. Participant responses were pooled and analysed by the research group, then circulated again as ‘feedback’ in a subsequent round [6].
Consumer focus groups
The focus group discussions (FGD) sought participants’ views on key research areas for PC, identifying the unmet needs of people with PC and inviting comment on initiatives that would be most helpful to patients and their carers.
We used FGDs to ascertain consumer views about PC research, as the flexible nature of questioning allows the exploration of the wide range of opinions and attitudes held by participants [22]. We could not identify previous reports describing ways to engage with PC patients around research prioritisation, likely related to the difficulty of carrying out research with people who suffer from a highly lethal cancer.
Conducting FGDs by telephone, rather than in the traditional face-to-face format was dictated by limitations in health status and mobility imposed by a PC diagnosis and allowed patients with advanced cancer and their carers to become engaged in the priority-setting process with minimal disruption to their lives. One discussion group comprised patients with a PC diagnosis, the other involved current and bereaved carers of patients with PC. Separating the groups allowed discussions to focus on issues specific to each group and how research could address them.
The aim of the research was outlined in a newsletter of the Pancreatic Cancer Support Project, distributed to people affected by PC. Interested potential participants were invited to register their interest in participating and participants were assured of confidentiality of the information.
The facilitated discussions were digitally recorded and subsequently transcribed. Two experienced telephone support group counsellors facilitated the discussions and an independent qualitative researcher played an observer role.
Nominal group process
The aim of the nominal group was to elicit participants’ views on PC research priorities most likely to accelerate progress and to develop consensus around a final list of Australian PC research goals. Participants included clinicians (medical and radiation oncologists, surgeons, and palliative care physicians), pathologists, basic science researchers, epidemiologists, and nurses.
The process comprised three steps: the identification of new research directions and issues, a discussion about key research directions identified through group work and the refining and tabling of the final research goals.
The second step involved each participant selecting and ranking their top three research priorities, while taking into account their feasibility within the Australian context. This involved consideration of existing Australian research capacity, availability of resources and the potential of the identified priorities to deliver a competitive advantage to Australian researchers. The ranked research priorities were subsequently discussed within the group and grouped into categories.
Results
Literature review
The literature review focussed on five key areas of research activity identified by the NCI Pancreatic Cancer Progress Review Group: [23] (1) disease aetiology and prevention, (2) early detection and screening, (3) PC diagnosis, assessment, and staging, (4) PC treatment and (5) palliative and supportive care.
Expert interviews
The expert group recommended a focus on the following research directions: developing better imaging and screening technologies, identifying genes that can be targeted by therapy, focusing on drug discovery, developing new animal models for PC, encouraging multidisciplinary research collaborations, emphasising epidemiological research and developing well-annotated clinico-pathological databases. Australian researchers were perceived to have a competitive advantage in database development and developing research networks that could advance PC research (see Table 1).
The Delphi process
In Round 1, participants were invited to contribute up to ten issues they believed could change outcomes in PC in the Australian context (n = 22, 73% response rate), with responses collated and circulated to the group in Round 2. As a substantial and very diverse number of research priorities (134) were identified, a group of PC experts devised ‘research categories’ enabling the grouping of similar or related items.
In Round 2a, the list derived from Round 1 and grouped by specific research category was circulated. Participants (n = 19, 63% response rate) were invited to confirm whether the process accurately captured their suggested priorities and invited to add (or edit) the statements to better encapsulate their views. These responses were used to inform Round 2b, where participants (n = 20, 66% response rate) were asked to nominate their five top priority areas from the final list. At that stage, research priorities selected by fewer than 15% of respondents (fewer than two participants) were excluded, resulting in 14 research priorities progressing to Round 3.
In Round 3, the relevance of these research priorities was ranked using a Likert scale, from 1 (research unimportant) to 5 (research very important); 25 respondents took part in this round (83% response rate).
Results of this round were circulated to participants in the fourth and final round, ranked in order of their importance, as identified in Round 3. Six research priorities received an average score >4 points, with the remaining eight priorities averaging between three and four points. The six highest ranking priorities and their average score are listed in Table 2.
Consumer focus groups
A range of issues were identified by both carers and patients as priorities for PC research. Four main themes were identified, related to the following: PC diagnosis, research into disease causation, better consumer and practitioner information about PC, and the quality of pancreatic cancer care. Different subthemes were identified and the information structured using a spreadsheet format. A summary of issues is presented in Table 3.
There was concordance between the patient and the carer group as to research priorities in PC, with most recommendations drawn from individual experiences. Specific research recommendations included the following: researching better modalities to deliver chemotherapy, reducing the deleterious effects of pancreatectomy on nutrition, finding simpler and faster screening tests, developing targeted screening for people at increased risk of PC, and exploring the role of genetic testing in disease prevention.
Nominal group process
New research issues and challenges identified by participants included administrative difficulties in the research application process, the impact of an increased use of complementary and alternative medicines, the need to distinguish issues germane to research from health delivery issues, to develop suitable trials for patients with advanced PC, and expediting the process of clinical trial selection for this group. In the final step, the most frequently listed research priorities were merged and refined into seven research goals (see Table 4).
The nominal group recommended the publication and dissemination of these research goals and suggested that future grant applications should identify specific research challenges and how to address them.
Discussion
This paper summarises a process of research prioritisation for a cancer which so far has eluded the attention of funding bodies, despite the significant disease burden it imposes. We used a combination of methods for defining research priorities, utilising the outputs of two consensus development methods (Delphi and nominal group processes) and focus group discussion to arrive at a list of “actionable” research goals specific to Australian PC research. The reason for combining several consensus building methods was to rule out that findings being influenced by the method chosen, although previous studies comparing the Delphi method to nominal group found that both methods performed equally well for predictive purposes [24]. In our study, research priorities generated through the Delphi process and the NGT were very similar and echoed consumer priorities identified through FGDs, validating our approach. Using a Delphi panel, a modified Delphi panel, and a survey-based panel, Brown et al. [25] identified medical conditions for which the risk of hospitalisation is related to ambulatory care access and found that consensus techniques had good predictive ability in this context.
A retrospective review of pancreatic cancer cases reported to a Cancer Registry in Burgundy (France) made similar recommendations regarding future PC research: focus on a better understanding of genetic and environmental causes of PC, on mechanisms involved in its development and on the role of diet in PC risk identified; key challenges related to detecting early/asymptomatic disease and improvements in therapy to improve outcomes [26].
Given the variety of opinions that may exist when a diverse group considers a highly technical and polarising topic, formal consensus approaches (the Delphi method, the nominal group process, and the consensus development conference) are increasingly used to make complex decision in medicine and health [27]. This is based upon the finding that in situations where the evidence limited, unclear or results diverge widely, consensus methods can provide “a bridge between clinical reasoning and clinical research” [28].
Traditionally cancer research priorities have been determined using a “group” model of decision-making, where options are advocated for by spokespersons and a judgement of quality and merit made by the group. While a wide range of experience and knowledge of group participants presents distinct advantages for group models of work [24], the output of large committees may be subverted by domineering personalities, or because group members may be unwilling to take a position before all facts are known, to change it once it is stated publicly, or to publicly contradict senior participants [29].
Our group chose consensus development methods in preference to committee or expert reference panels, due to their perceived limitations and based on the proven track record of formal consensus methods in achieving research prioritisation in the health arena [16, 17, 19, 30–34]. It was encouraging to note also that four of the seven research priorities identified by the nominal group arose also from the consumer FGDs recommendations.
Discussions in the planning phase of the research led to a decision not to include consumers in the Delphi process, to safeguard against the potential “lay disempowerment” that may arise when bringing together experts and consumers. Qualitative researchers have questioned whether the notion of “empowerment” reflects participants acquiescing researcher’s views, rather than empowerment to express their own views [35]. While studying Australian research priorities in mental health, Griffiths et al. [36] found significant differences in research priorities between the committees responsible for evaluating research grants and consumer and carer groups. We wanted to be able to identify specific consumer and researcher perspectives on research priorities before merging them into a final set of recommendations through the nominal group process.
A telephone-based system can present specific challenges for the conduct of FGDs, as important non-verbal cues may be overlooked in this setting. A key element in obtaining quality data and providing a good experience to participants is to ensure that moderators are experienced in eliciting the views of severely ill patients. In this project, the FGDs were conducted by a pair of specially trained telephone counsellors working with people with cancer. They worked closely with a qualitative researcher with expertise in conducting FGDs, to ensure that participants had a positive experience and that all participants had the opportunity to have their views heard. The FGDs produced a large amount of key information relatively expeditiously and in a way that allowed the participation of consumers who were unwell and would not otherwise have the opportunity to have their voice heard.
Study limitations
While the Delphi process involved a panel of 27 experts, their heavy workloads precluded their participation in every round, so response rates ranged between 63 and 85% per round. However, the inclusion of multiple rounds—including a final opportunity to dispute or add to the group’s consensus—enables us to feel confident that the 14 priorities identified reflect the views of the entire panel.
We appreciate that the findings of the FGDs cannot be used to make statements about what all those affected by PC think about key research priorities, but the level of concordance reached independently by patients and carers would suggest that to some extent at least these results are generalisable to the larger group. While focus groups are intended to encourage participants to talk with each other, rather than answer the moderators’ questions, this presented specific challenges for a telephone mode of delivery. Using experienced support group moderators was essential in ensuring group interaction and to provide an in-depth understanding of consumer’s views.
The FGD recommendations, together with the results of the Delphi process were discussed in a nominal group. This enabled the list of research priorities to be discussed in a face-to-face setting, alleviating some of the criticisms levelled at the Delphi methodology, such as the potential of manipulating the consensus, unreliability of the process, and doubts as to what represents acceptable level of consensus [37, 38].
The nominal group defined and ranked seven research directions for Australian PC research. We critically summarised the findings of the consensus process and fed them back to the participants for comment. The priority list was circulated to researchers in the field, inviting applications for a new funding scheme—the Innovator Grants. The grants were open to all applicants who addressed the research priorities and the application process was simplified, in keeping with suggestions received from nominal group participants. Twenty-three applications were received and six were funded through the new scheme, closing the loop from research prioritisation to implementation.
Conclusion
This prioritisation exercise provided a much needed “road map” for research prioritisation in pancreatic cancer, providing valuable information to researchers applying for PC research grants. Additionally, the consumer FGDs provided insights into what patients and their carers see as critical for PC research: better and earlier diagnostic modalities, improvements in understanding disease aetiology, high quality information provision about all aspects of the disease and its treatment and devising better ways in communication between doctors and patients. It was encouraging to note that many consumer research priorities rose to the top of the priority list and will hopefully provide new research insights in PC.
References
AIHW (Australian Institute of Health and Welfare) and AACR (Australasian Association of Cancer Registries) (2007) Cancer in Australia: an overview, 2006. Cancer, AIHW, Canberra
Postier RG (2003) The challenge of pancreatic cancer. Am J Surg 186(6):579–582
Tracey E, Alam N, Chen W, Bishop J (2008) Cancer in New Wouth Wales: incidence and mortality 2006. In: NSW CI (ed) Cancer in New South Wales. Cancer Institute, Sydney
Pancreatic Cancer Progress Review Group (2001) Pancreatic cancer: an agenda for action. Report of the pancreatic cancer progress review group. In: Institute NC (ed), National Cancer Institute
Ronald S (2008) Hirshberg Translational Pancreatic Cancer Research Laboratory, Translational Research, Los Angeles
Dalkey N, Helmer O (1963) An experimental application of the DELPHI method to the use of experts. Manag Sci 9(3):458–467
Biondo PD, Nekolaichuk CL, Stiles C, Fainsinger R, Hagen NA (2008) Applying the delphi process to palliative care tool development: lessons learned. Support Care Cancer 16(8):935–942
Gagliardi AR, Lemieux-Charles L, Brown AD, Sullivan T, Goel V (2008) Barriers to patient involvement in health service planning and evaluation: an exploratory study. Patient Educ Couns 70(2):234–241
Hermens RP, Ouwens MM, Vonk-Okhuijsen SY, van der Wel Y, Tjan-Heijnen VC, van den Broek LD et al (2006) Development of quality indicators for diagnosis and treatment of patients with non-small cell lung cancer: a first step toward implementing a multidisciplinary, evidence-based guideline. Lung Cancer 54(1):117–124
McGory ML, Shekelle PG, Ko CY (2006) Development of quality indicators for patients undergoing colorectal cancer surgery. J Natl Cancer Inst 98(22):1623–1633
Roddy E, Zhang W, Doherty M, Arden NK, Barlow J, Birrell F et al (2005) Evidence-based recommendations for the role of exercise in the management of osteoarthritis of the hip or knee—the MOVE consensus. Rheumatology (Oxford) 44(1):67–73
Greenberg A, Angus H, Sullivan T, Brown AD (2005) Development of a set of strategy-based system-level cancer care performance indicators in Ontario, Canada. Int J Qual Health Care 17(2):107–114
Elwyn G, O’Connor A, Stacey D, Volk R, Edwards A, Coulter A et al (2006) Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ 333(7565):417
Bond S, Bond J (1982) A Delphi survey of clinical nursing research priorities. J Adv Nurs 7(6):565–575
Chang E, Daly J (1998) Priority areas for clinical research in palliative care nursing. Int J Nurs Pract 4(4):247–253
Cohen MZ, Harle M, Woll AM, Despa S, Munsell MF (2004) Delphi survey of nursing research priorities. Oncol Nurs Forum 31(5):1011–1018
Fochtman D, Hinds PS (2000) Identifying nursing research priorities in a pediatric clinical trials cooperative group: the pediatric oncology Group experience. J Pediatr Oncol Nurs 17(2):83–87
Hinds P, Norville R, Anthony L, Briscoe B, Gattuso J, Quargnenti A et al (1990) Pediatric cancer nursing research priorities: a Delphi study. J Pediatr Oncol Nurs 7(2):51–52
Monterosso L, Dadd G, Ranson K, Toye C (2001) Priorities for paediatric cancer nursing research in Western Australia: a Delphi study. Contemp Nurse 11(2–3):142–152
Efstathiou N, Ameen J, Coll AM (2007) Healthcare providers’ priorities for cancer care: a Delphi study in Greece. Eur J Oncol Nurs 11(2):141–150
Broomfield D, Humphris GM (2001) Using the Delphi technique to identify the cancer education requirements of general practitioners. Med Educ 35(10):928–937
Krueger R (1988) Focus groups: a practical guide for applied research. Sage, London
Pancreatic Cancer Progress Review Group (2001) Pancreatic cancer: an agenda for action. Report of the pancreatic cancer progress review group. http://planning.cancer.gov/pdfprgreports/2001pancreatic.pdf, National Cancer Institute
Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CF, Askham J et al (1998) Consensus development methods, and their use in clinical guideline development. Health Technol Assess 2(3):i–iv 1–88
Brown AD, Goldacre MJ, Hicks N, Rourke JT, McMurtry RY, Brown JD et al (2001) Hospitalization for ambulatory care-sensitive conditions: a method for comparative access and quality studies using routinely collected statistics. Can J Public Health 92(2):155–159
David M, Lepage C, Jouve JL, Jooste V, Chauvenet M, Faivre J et al (2009) Management and prognosis of pancreatic cancer over a 30-year period. Br J Cancer 101(2):215–218
Fink A, Kosecoff J, Chassin M, Brook RH (1984) Consensus methods: characteristics and guidelines for use. Am J Public Health 74(9):979–983
Cross H (2005) Consensus methods: a bridge between clinical reasoning and clinical research? Int J Lepr Other Mycobact Dis 73(1):28–32
Turoff M (1975) The policy Delphi. In: Linstone HA, Turoff M (eds) The Delphi method: techniques and applications. Addison-Wesley Publishing Co, Boston, pp 84–101
Browne N, Robinson L, Richardson A (2002) A Delphi study on the research priorities of European oncology nurses. Eur J Oncol Nurs 6(3):133–144 discussion 145–7
Lee EH, Kim JS, Chung BY, Bok MS, Song BE, Kong SW et al (2003) Research priorities of Korean oncology nurses. Cancer Nurs 26(5):387–391
McIlfatrick SJ, Keeney S (2003) Identifying cancer nursing research priorities using the Delphi technique. J Adv Nurs 42(6):629–636
Robert G, Milne R (1999) A Delphi study to establish national cost-effectiveness research priorities for positron emission tomography. Eur J Radiol 30(1):54–60
Rudy SF, Wilkinson MA, Dropkin MJ, Stevens G (1998) Otorhinolaryngology nursing research priorities: results of the 1996/1997 SOHN Delphi survey. ORL Head Neck Nurs 16(1):14–20
Opie A (1992) Qualitative research, appropriation of the ‘other’ and empowerment. Fem Rev Spring:52–69
Griffiths KM, Jorm AF, Christensen H, Medway J, Dear KB (2002) Research priorities in mental health, part 2: an evaluation of the current research effort against stakeholders’ priorities. Aust N Z J Psychiatry 36(3):327–339
Rennie D (1981) Consensus statements. N Engl J Med 304(11):665–666
Williams PL, Webb C (1994) The Delphi technique: a methodological discussion. J Adv Nurs 19(1):180–186
Acknowledgments
This project was funded by The Cancer Council NSW and was the product of a collaboration between CCNSW and the NSW Pancreatic Cancer Network (www.pancreaticcancer.net.au). We would like to thank Mr. John Friedsam for his contribution to the conduct of the focus group discussions and to all people affected by pancreatic cancer who participated in this study. Special thanks go to the large group of experts who shared their experience regarding pancreatic cancer research and its challenges and opportunities. This study was funded by the Cancer Council NSW.
Open Access
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Robotin, M.C., Jones, S.C., Biankin, A.V. et al. Defining research priorities for pancreatic cancer in Australia: results of a consensus development process. Cancer Causes Control 21, 729–736 (2010). https://doi.org/10.1007/s10552-010-9501-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10552-010-9501-1