Background

The COVID-19 pandemic has brought significant attention on public health (PH) systems globally. PH systems can be broadly defined as the complex networks of governmental organizations, departments, agencies and associations which are planning, managing and delivering PH services [1, 2]. The COVID-19 crisis has accelerated already engaged critical reflections on PH systems for the 20th century on their infrastructure, scope, processes, outcomes and performance, with Canada being one country particularly active in these reflections since the SARS crisis [3]. Several provincial, territorial, and federal government actors are currently undertaking consultations and reflections to prepare the future of PH systems post-COVID-19 [2, 4,5,6]. These consultations are yet to produce their full effects in terms of reforms to the Canadian PH systems, as they compete with emerging areas of focus in public health (e.g., public health and climate action) or in healthcare (e.g., funding and surgery backlogs) [7, 8].

Public health systems are understandably complex [9]. In Canada, differing PH responsibilities lie at the federal, provincial/territorial, regional and municipal levels [10]. PH functions include population health assessment, health surveillance, disease and injury prevention, health promotion and health protection [11]. Daily PH interventions at a local or regional level include health needs assessments, direct actions targeted towards populations, indirect actions targeted towards third parties involved in public health response such as support, collaboration and advocacy, as well as planning and evaluation [12]. With pandemic inquiries currently examining PH systems across Canada, multiple priorities will be proposed to allow for concrete actions for renewal of PH systems.

Health systems are particularly known to be slow and incremental to reform, before windows of opportunity for large-scale, substantial reforms open [13]. The COVID-19 pandemic provides this kind of window of opportunity for major reform to PH systems, with both opportunities and risks. Collecting data from PH senior decision-makers in this setting, and using a method designed to build consensus where possible and identifying consistent needs across jurisdictions, is thus a particularly timely endeavor.

Given the underdeveloped field of PH systems and services research [14], and despite several policy reports on reforming PH systems, PH decision makers’ priorities on PH systems are seldom systematically investigated and documented. This study provides a first step in this direction and will help highlight priorities for both PH policies and intervention research. Our objectives are to understand PH decision makers’ priorities for PH systems reform across Canada, and ultimately generate evidence on PH decision-makers expectations for future PH systems’ reorganizations. Precisely, our study aims to identify if a consensus can emerge on priorities for public health financing, organization, interventions, and workforce across senior PH decision makers located in several jurisdictions and organizations across Canada.

Methods

Research design

We developed and administered a Delphi survey to collect data from senior PH decision makers. The Delphi approach is a method for organizing and sharing opinions among a panel of experts with the aim to identify areas of consensus [15,16,17]. Experts are asked to make judgments, usually via a rating of importance, on a list of ideas which are called propositions in the Delphi terminology. The Delphi approach is iterative, and the rating of importance is repeated by one expert on several occasions called rounds. Because the purpose is potential consensus building, the experts are presented with the ratings of the group between each round [18,19,20] and asked to reconsider their ratings in the next rounds based on this new information. We used a type of real-time Delphi in an online survey characterized by participants being able to participate and edit their responses anytime during a specific timeframe [21].

The first step of a Delphi is thus to develop a list of propositions. We started by conducting a literature review to identify key components of public health infrastructures [22,23,24] and we consulted with a working group of PH experts, including PH academic, PH practitioners and clinicians, and PH managers, to develop an initial list of propositions summarizing important characteristics of PH infrastructure in Canada. We then refined our list of propositions in three consultations with this working group of PH experts and ran a pilot test of the first round of the Delphi survey with 3 participants. We then aggregated our proposition lists from 16 to 9 propositions due to overlap in themes. The 9 propositions cover the domains of PH financing, organization, workforce, and interventions, all well-identified as key dimensions in the literature of PH systems [22,23,24]. These specific propositions were also consistent with recent PH systems reorganizations in various Canadian jurisdictions [1] and with the key messages that emerged in the recent Canadian Chief Public Health Officer’s report [2]. Additionally, we added an option for participants to provide open-ended comments on up to three priorities, to be analyzed separately. We included this option to provide nuance to the findings of the Delphi.

The second step was to determine the number of Delphi rounds, a debated issue in the literature [20, 25]. On one hand, some recommend that the appropriate number of rounds should be determined once the panelists show a level of stability in their individual responses [19, 20, 25]. On the other hand, some scholars argue that practical constraints such as time, cost, and expert availability require a set number of Delphi rounds. Since three rounds are generally considered sufficient to reach group consensus [18, 20, 26], we elected for a fixed three-round data collection given the expected constraint on PH experts’ availability.

The Delphi process was conducted in an online format, as all real-time Delphi, and owing to the COVID-19 pandemic context. Note that online Delphi performs similarly to in person [18, 21, 27, 28].

Survey instruments

Participants were prompted to identify their preferences for changes to public health financing and structures in order to determine emerging priority areas and help inform the future changes for public health systems in Canada. They were asked to rate the nine propositions on a 5-point Likert scale (“not at all important”, “slightly important”, “moderately important”, “very important”, “extremely important”). Propositions in the financing domain include provincial or territorial PH budget, sources of PH financing, and time frame for spending. For the domain of PH organization, propositions concern centralization or decentralization of public health system, integration of public health with other health sectors, and the creation of public health structures with specialized public health functions. Further, we assess participants’ opinions regarding the disciplinary skill-mix of PH human resources. Our last domain covers PH interventions related to the COVID-19 pandemic and those which are not related to the COVID-19 pandemic. We provide the full description of our questionnaire in the Supplementary Materials. In Round 2, we show individual and aggregate responses to each participant and invite them to re-rate the nine propositions with the possibility to re-consider their initial responses. They receive the same instructions during the Round 3.

The Delphi survey was developed online using Qualtrics. The survey was initially developed in English and then translated in French by our bilingual team members: one researcher translated the survey to French, and another translated it back to English to ensure equivalence. Only minor adjustments were made to the initial French translation. An important feature of our survey design is that, after round 1, each respondent receives an individualized survey. Indeed, in round 2, each respondent receives the results summarizing both their own response and the overall responses on the importance of each of the 9 propositions at the end of the first round. This is with the view to minimize survey recall bias and inform the respondent of the collective priorities. With this new information, the respondents are invited to re-rate the 9 propositions with the possibility to re-consider their initial responses. The same process applies in Round 3, displaying the individual and group responses of the second round.

Participants’ recruitment

The literature on the number of participants required in a Delphi exercise suggests many acceptable ranges, from 5 to 50 participants [29,30,31]. The lower end of these ranges have been used in empirical studies [32,33,34] and most recently, Vogel et al. [35] argued that 12 respondents is a sufficient minimum. In line with Belton et al. [18] and Rowe and Wright [20], we invited 86 PH decision makers to participate in the Delphi survey. We aimed for 15 to 20 respondents in the last round and our number of first invitations account for potentially high initial non-response and for some attrition between rounds.

Our target population was of individuals in senior position in a Canadian public health organization (e.g., provincial public health organization, local public health unit) or a health organization with a public health mandate (e.g., Ministry of Health, regional health authority). The rationale for focusing on senior decision-makers was for them to be able to directly influence or act on priorities for PH infrastructure and to address gaps in the literature in understanding this perspective. This rationale led us to exclude academics from our target population. Senior roles include medical officers of health, senior manager, or senior policy advisors in these organizations. We aimed for a national coverage and invited respondents from all provinces and territories. We invited respondents working in provincial/territorial and regional authorities to account for the multi-level nature of PH systems. We built a directory from publicly available information, supplemented by input from knowledge users for those harder to locate. We adopted a non-probability purposive sample strategy to ensure invited participants met our inclusion criteria.

Respondents were invited to participate by email sent to their professional address in April 2021. To increase participation, we asked our knowledge users to champion the Delphi in their province/territory by sending a pre-invitation email highlighting our invitation was coming. The data collection occurred from April to June 2021, only those who received the individual invitation could access their personalized survey, and only respondents who replied to the preceding round received invitation for the next one.

Reaching consensus

There is no agreed position in Delphi literature concerning the way that researchers must define and operationalize consensus among the participants [19, 26, 36,37,38]. Some recommend using both stability and consensus on a round-by-round basis and continuing until acceptable levels of both are achieved [18, 19]. Consensus criteria can include the same or similar opinion being reported by 70% [35, 36, 39, 40], 75% [29, 41] or 80% of experts [31, 42]. In this paper, we define consensus as a minimum of 70% of participants’ agreement on the importance of the proposition. We selected this lower bound given the heterogenous participants coming from various organizations and geographic levels with goals potentially not aligned. Participants are deemed to agree on a proposition’s importance if they are at least 70% to judge it as very/extremely important. They agree on a proposition being not important if they are at least 70% to rate it as not/slightly/moderately important.

Data analysis

We perform descriptive statistics to describe respondents’ characteristics and group responses to each proposition in all three Delphi rounds. Analyses are conducted using STATA SE 16. Open-ended comments are synthesized using thematic analysis [43]. Themes were organized through discussion, summary tables, and mapping.

Results

Participants description

Of the 86 participants invited to take part the Delphi study, 25 experts completed Round 1 (29% response rate), 19 of 25 completed Round 2 (76% retention rate) and 18 of 19 completed Round 3 (95% retention rate). The dropout rate between each round is thus consistent with the health related Delphi literature [35, 39]. Table 1 presents the demographic characteristics of participants in each round. At the beginning of the survey 20% of participants received the French version of the questionnaire while 80% participated in English. Respondents of all the parts of Canada were represented, roughly proportional to the population and with slightly fewer responses from the territories, as expected. Most participants are non-MOH executive, who responded consistently across rounds, while the share of CMOH and MOH decreased from the first to the last round. Respondents are slightly more represented in Ministries of Health, and relatively evenly spread between Public Health Agencies and Regional Health Authorities.

Table 1 Characteristics of the participants
Full size table

Delphi main findings

Table 2 presents a summary of the importance ratings for all Delphi propositions across the three rounds of data collection. Recall ratings as “very important” or “extremely important” are aggregated in the “important” category, while the other levels of the Likert scale fall in the “not important” aggregated category, and we provide more details of the full distribution of the ratings for Round 3 in Fig. 1. The numbers in Table 2 reports the percentage of respondents who judge the proposition as important or not important, and figures in bold indicate that the consensus has been reached for the proposition.

The number of propositions where consensus was achieved increased for several propositions from Round 1 to Round 3. In Round 1, consensus was achieved for 2 of the 9 propositions. In Round 2, consensus was achieved for 4 of the 9 propositions and this rose to 6 out of 9 in Round 3. By Round 3, consensus was achieved on the importance of: public health budget (89% judge it important in round 3); time frame for spending (72%); public health structures with specialization of public health functions (83%); public health interventions related to the COVID-19 pandemic (78%); and PH interventions not related to the COVID-19 pandemic (e.g., environmental health protection, prevention and control of other infectious diseases) (89%). The proposition pertaining to the source of PH financing is the only one where the consensus led to a rating of non-importance (78% of respondents judge it unimportant).

Turning to Fig. 1, it appears that among propositions where consensus was reached, majority judgment of extreme importance was only attained for PH budgets and PH interventions not related to COVID-19. The other consensual propositions (time frame for spending, PH specialization, COVID-19 PH interventions) were rated as very important.

Table 2 Responses to the propositions across the three rounds
Full size table
Fig. 1
figure 1

Participants’ responses during the third round of the Delphi survey

Full size image

Synthesizing open-ended comments provided

Free-text suggestions were solicited during Round 1 of the Delphi process and most participants provided comments. Most of the comments touched on governance and decision-making, the importance of financing/resourcing and human resources. Less common comments related to data and information systems. In sum, in addition to the domains covered in the survey (i.e., financing and workforce), two other themes emerged (i.e., governance and information system).

“Ongoing committed long-term funding rather than responsive funding to address each emerging public health crisis. This allows long-term planning and optimization as well as better prevention.”

Regarding PH financing, survey respondents noted the need to increase (and protect) the PH budget over the long-term, to increase funding for and consolidation of public health laboratories, to allow for flexibility within PH budgets in terms of the different PH functions, and to consider equity in the funding formulas to reflect differences in local populations’ health and vulnerabilities. Key priorities noted within the PH workforce domain included the need to expand and “upskill” the PH workforce and investing in training and increased compensation.

“The expert public health workforce has been demoralized during the pandemic response, decimating available expertise. Investment in training, improved compensation, and most importantly basic respect for front line public health expertise is urgently required.”

Many of the open-text comments related to public health governance. Some of the comments touched on aspects of governance and decision making around financing and workforce, such as the need to develop a strategy and specific objectives for PH to use to then clarify and justify the budgets and resources, as well as the need to develop a workforce strategy. Multiple individuals commented on the need for increased clarity of public health objectives and the inherent challenges given the nature of the work. Some reflected on the need to prioritize more non-communicable disease work (over communicable disease) and continue to work towards prioritizing prevention/promotion over treatment, an aspect covered in the survey. Other aspects of governance noted in the comments included maintaining or increasing the independence of PH actors, with some noting the importance of maintaining independence from the acute care system, as well as supporting intersectoral partnerships and including multiple stakeholders (e.g., from social sectors) in decision making. Some indicated a lack of respect for PH expertise, particularly that of local/regional PH leadership.

“Governance of public health and its independence from the administrative functions of government. Accountability of public health with respect to funding, organization of services and personnel.”

A last and smaller group of comments pertained to the importance of data management and information systems. They reflected on epidemiologically driven nature of public health and the need for improved availability of data for PH to continue advancing its work and the science.

Discussion

Key findings

Using a Delphi methodology, we investigated PH senior level decision-makers priorities for the future of PH infrastructure in Canada. The development of the Delphi survey in close consultation with PH knowledge users brought us to focus on PH financing, organization, workforce, and interventions. The relevance of this focus was recently corroborated by priority research areas identified by the CIHR Institute of Population & Public Health [44]. Consensus was reached on six out of nine propositions. One of the most important findings is the immediate consensus in the first round of the Delphi, and agreement on extreme importance among PH decision-makers on the importance of PH budget and PH non-COVID-19 interventions. Several have observed cuts in PH budgets [45, 46], even if there are variations across Canada [47]. The need for sufficient financial means to support their work is thus a top priority, and PH decision-makers are also mindful that the COVID-19 pandemic shall not crowd-out all the other necessary PH interventions. With regards to non-COVID public health interventions needed to be prioritized, multiple comments specifically pointed at the importance of upstream and health promotion interventions. Indeed, public health institutions have faced a recurring challenge in simultaneously addressing population health needs and public health emergencies [6, 48]. Nevertheless, PH decision-makers also agree that PH interventions designed to tackle the COVID-19 pandemic are important, perhaps recognizing that the implications of the pandemic will be far reaching.

Among other propositions reaching consensus, the time to spend the PH budget is important but the source of financing is not. While the source of financing could reflect a division of power in the way the money is spent, most jurisdictions receive funding mostly from provincial and federal governments. The main exception is the province of Ontario, where the municipal level provides a nonnegligible share of the funding [49]. Hence, the little variation they experience in the source of financing may explain why PH decision-makers from across the country do not see this as a priority. On the other hand, the time frame to spend the PH budget may be related to the long-term planning and implementation horizon of many PH interventions. Creating PH structures with more specialized PH functions also reached consensus. Most provinces and territories do not have regional PH expertise centers (e.g., BC Centre for Disease Control, Public Health Ontario, Institut National de la Santé Publique du Québec) [49,50,51,52], and the consensus may reflect a desire to protect or expand those centers.

Where consensus was not reached, it was for the centralization/decentralization of PH, PH integration and the skill-mix of the PH workforce. Canadian health systems, including PH, have increasingly recentralized from the mid-2000s, but there is no clear evidence on the right level of centralization/decentralization [53]. Integration of PH with other healthcare sectors, particularly primary care, is recognized as a way to improve population health [54]. However, only some provinces like Quebec have been moving in this direction [55]. Moreover, while there are clearly established core competencies for PH in Canada [56], little is known on the size and composition of the Canadian PH workforce [49, 51]. The dearth of evidence on these matters may help explain why consensus was not reached, even though some comments in the open text pertained to skill-mix.

Overall, free-text comments provided by participants in the first round of the Delphi process provided useful and complementary insights, particularly on the critical importance of public health governance. PH governance can be understood as “the ways in which public, non-governmental, or private actors work together to support communities in preventing disease and achieving health, wellbeing, and health equity” [57]. Functions of PH governance include resource stewardship [58], which emerged as an important theme in the comments.

Strengths and limitations

Our study has several strengths. We focused on PH senior decision makers rarely involved in anonymous, consensus-building processes. This approach should limit issues that may occur in group consultations where authority, personality or reputation may affect priority-setting. Obtaining the perspective of individuals likely to be making or influencing policy decisions is an important contribution. We further achieved a good coverage of multiple level of responsibilities, from local to provincial. Furthermore, given its reasonable time-demand, the online Delphi may have been the most realistic strategy to survey PH decision-makers during a pandemic. We used a bilingual survey, ensuring representation of linguistic minority in Canada, and covered the country coast to coast to coast.

Our study also has limitations. We obtained an uneven initial response rate across jurisdictions, which may be explained by differential timing of COVID waves across the country. However, we achieved high retention rate among the respondents. We aimed for a short list of propositions to maximize the survey uptake, hence not all areas of PH infrastructure were addressed. Further, we did not change the propositions between each Delphi round. We aimed to mitigate this by offering the option for open-ended comments, which helped reveal the importance of public health governance as well as data management and information systems. Lastly, the formulation of our propositions did not allow us to know if changes related priorities identified by participants were already being implemented.

Future research could delve into those priorities with more extensive qualitative methods to consolidate and refine concrete policy options. This could be done by expanding the investigation to larger PH community (e.g., academics, PH staff). It would also be useful to know if these priorities resulted in actual system changes. Additionally, a quantitative study could help clarify the potential trade-offs among priorities.

Conclusion

Public health systems renewal is now at the top of many policy agendas. Ensuring that their redesign is informed by public health communities will not only increase the readiness of the systems for future public health challenges, but they will also increase their acceptability by those in charge of transforming the public health systems. Despite the difficulty of obtaining participation from public health leaders in the middle of the COVID-19 pandemic, we attained the required number of responses for the Delphi methodology and achieved a geographic coverage of Canada. The top priorities of PH senior decision-makers are PH budget and ensuring non-COVID PH interventions are not forgotten because of the pandemic. Future research can help assess if and how PH decision makers are willing to accept trade-offs between these priorities.