Introduction

The 2019 SARS-CoV-2 (COVID-19) pandemic has caused unprecedented disruption and challenges worldwide. In response, broad public health surveillance and response measures have been implemented to minimize transmission and protect individuals susceptible to severe disease while limiting societal disruption. Despite highly effective vaccines, COVID-19 continues to spread globally, resulting in the prolonged implementation of stringent public health measures.

The broad impacts of the COVID-19 pandemic and the public health response measures have greatly affected everyday life, including physical, mental, social, and economic well-being (Douglas et al., 2020; Wang et al., 2020; World Health Organization, 2020). These impacts have further exacerbated disparities in health outcomes and determinants of health and vulnerabilities within our healthcare system. Many segments of the population that already experience inequities, including people with low socioeconomic status, visible minorities and marginalized groups, young adults, and families with children, have been disproportionally affected (Munasinghe et al., 2020; Samji et al., 2021; Wang et al., 2020).

The wide-reaching societal impacts and interventions related to the pandemic have driven the need for dynamic population health surveillance to understand and address the societal consequences of the pandemic. Large-scale representative population health surveys can provide reliable insight into individuals’ experiences and inform the public health response and societal changes needed to support the health and well-being of the population and reduce health inequities (World Health Organization Regional Office for Europe, 2020). Due to the significant variation in the transmission of SARS-CoV-2 and public health response strategies across Canadian and international jurisdictions, there was a need for a comprehensive and representative survey to inform public health services and pandemic response measures within British Columbia (BC), Canada.

The BC COVID-19 Survey on Population Experiences, Action and Knowledge (SPEAK) measured the populations’ perceptions of risk, acceptability of the public health response and recovery measures, and the broader impacts of the COVID-19 pandemic at local, regional, and provincial levels. The initial survey (round one) assessed BC residents’ experiences during the early stages of the pandemic to inform ongoing public health measures and assess the unintended consequences. The second survey (round two) was conducted a year later to assess the changes in behaviours and experiences since the early phase of the pandemic; understand barriers to vaccination; inform health and social and economic investment during the pandemic recovery; and assess inequities across different population groups.

This paper provides an overview of the methods used to develop the population health surveys, key findings, and how these findings have informed public health initiatives during the COVID-19 pandemic in BC.

Methods

Study design and data collection

An observational cross-sectional study design assessed the experiences of the COVID-19 pandemic among the adult population in BC, Canada, at two specific time points: 12 May–31 May 2020 and 8 April–9 May 2021.

Survey development

The two SPEAK surveys were designed and implemented using the same methodology to answer questions relevant to specific pandemic stages, sharing many core questions and enabling cross-sectional comparisons over time. A working group was formed with public health leaders across BC, including provincial and regional organizations representatives, to share knowledge and local perspectives.

Targeted literature reviews and environmental scans provided the theoretical basis for the domains of interest, survey objectives, and questions. Key survey domains reflected the survey’s overarching goals, and multiple questions were selected or developed to represent each domain. The Social Determinants of Health Model (Whitehead & Dahlgren, 2006) informed the development and selection of the domains and questions. This model is relevant to understanding health inequities, public health priority areas, and the unintended impacts of the pandemic and public health response measures.

The initial survey covered eight domains: socio-demographics; COVID-19 response, testing, and prevention; experience; risk and protective factors; healthcare; social; economic; and resiliency. The second survey encompassed ten domains: the eight from round one and the added domains of vaccine and adaption. Survey questions were primarily selected or adapted from publicly available or validated tools:

  • Statistics Canada (Statistics Canada, 2009, 2018, 2020a, 2020b, 2020c, 2020d, 2020e; Statistics Canada, Mexico’s Instituto Nacional de Estadística y Geografía (INEGI), & Economic Classification Policy Committee (ECPC) of the United States Office of Management and Budget, 2017);

  • Canadian and International Health Surveys (Carman et al., 2020; Johns Hopkins University, 2020; Ogilvie et al., 2021; United Kingdom Office for National Statistics, 2021; University of California Los Angeles (UCLA), 2004, 2020a, 2020b; Vancouver Coastal Health, Fraser Health, & University of British Columbia, 2020); and

  • World Health Organization (World Health Organization Regional Office for Europe, 2020).

Additionally, the working group created several novel questions. Questions were reviewed and selected relevant to public health priority areas. The round one survey consisted of 85 questions; 52 of the 85 were retained for round two, and a further 50 questions were added, resulting in 102 questions (Table 1). The questions added in round two evaluated attitudes related to vaccination, pandemic adaption, and other mental health and societal impacts. All questions were categorical, except for two open-ended questions to further explore respondents’ experiences. Aside from age, sex, and geographic indicators, all questions were optional and included a “prefer not to answer” response option.

Table 1 Composition of the BC COVID-19 SPEAK surveys
Full size table

Survey delivery and testing

Qualtrics (Qualtrics, 2020), an online survey tool, was used to deliver both surveys. A web-based survey was chosen over paper or telephone survey methods to facilitate cost-effective and rapid development, data collection, and analysis and reduce manual entry error. A call centre was also established for round one to assist individuals who needed support to complete the survey; this assistance was not offered in round two due to low uptake. The surveys were available in English and Simplified Chinese for round one, and French and Punjabi were added in round two. Language guides were available for both survey rounds in French, Punjabi, American Sign Language, Korean, Spanish, Vietnamese, Farsi, Arabic, and Chinese.

Pre-testing of the survey was conducted with members of the working group to evaluate face validity, comprehension, content, layout, and design. Technical aspects were tested to maximize accessibility and compatibility across most platforms (smartphones, computers, and tablets) and internet browsers. Completion times in English, French, Chinese, and Punjabi were assessed, averaging 10–20 min (round one) and 20–30 min (round two).

Participants, sampling, and recruitment

Both surveys’ target population encompassed all residents of BC aged 18 years and older. A non-probability quota-based sampling method was used, rather than probability random sampling methods, as it was the most time-efficient and inexpensive way to obtain the information required (Groves et al., 2009). To ensure that representative samples were obtained across different geographic regions of BC, sampling quotas were calculated for age, sex, income, education, and ethnicity (Appendix 1 and Appendix 2). The areas were defined by BC health administrative boundaries, using hierarchical categorization of data ordered from most to least granular: Community Health Service Area (CHSA), Local Health Area (LHA), Health Service Delivery Area (HSDA), Health Authority (HA), and the Provincial (BC) level.

Using Census data, sample size calculations were performed for each CHSA by age and sex (Statistics Canada, 2016). HSDA targets were determined by either the crude target (2% of the urban population or 4% of the rural population determined by CHSA population density rank) or the sample size based on the hypergeometric distribution with a 4% margin of error, whichever was larger (Appendix 1 and Appendix 2). Progress toward recruitment targets was monitored daily; however, outreach was limited due to pandemic response measures. In addition, 250,901 respondents from the initial survey who provided an email for follow-up were invited to participate in round two.

Statistical methods

Statistical analysis was performed using Statistical Analysis System (SAS version 9.4) (SAS Institute Inc, 2008) and R (version 3.6.2) (R Core Team, 2013) statistical software packages.

Data preparation

The data were cleaned to improve overall data quality. Duplicate surveys and those with missing age, sex, and geography data were removed. A minimum degree of progression through the survey was required for inclusion in the final analytical dataset. Cut-off points were determined by assessing the natural attrition points of survey progression. After review, a cut-off point for survey progression was selected at 31% for round one and 33% for round two. Data were also suppressed for geographical areas with more than 25% Indigenous population in accordance with Indigenous data governance practices.

Weighting

Post-collection statistical weighting was performed to minimize potential biases introduced by the study design and sampling methods and to ensure the results were representative of the BC population using Census data by geography (HSDA, LHA, and CHSA) based on age, sex, education, and ethnicity questions (Appendix 3). Stratifications were limited while optimizing representativeness across the survey region. The weighted values were calculated as percentages with corresponding 95% confidence intervals (CI). Coefficients of variation were calculated and estimates greater than 33.3% were considered unreliable and were suppressed.

Validation of sample

Several questions were derived from the Canadian Community Health Survey (CCHS) (Statistics Canada, 2018, 2020a). The CCHS is a large cross-sectional survey using a rigorous methodology and a probability random sampling method to provide representative health region–level estimates every 2 years. However, CCHS may be subject to selection bias, as it is conducted by phone in English or French. Comparisons between pre-pandemic indicators (non-communicable conditions, mental health, social connectedness, and lifestyle risk factors) from the 2017/2018 CCHS (Statistics Canada, 2018) and the SPEAK surveys were conducted to contextualize and assess the representativeness of the SPEAK survey samples during the pandemic.

Results

Sample population

In total, 394,382 (round one) and 188,561 (round two) individuals were included in the analytical datasets, providing a large and comprehensive sample of the BC population (approximately one in ten and one in twenty-five people aged 18 years and over residing in the province, respectively) (Table 2). A total of 250,901 round one participants who provided their contact details were invited to participate in round two; 148,452 (59.2%) responded. A total of 141,728 survey responses were included in the final dataset, providing longitudinal data to assess changes between the survey rounds at individual and population levels. The sample was weighted using the 2016 census data, and the unweighted and weighted samples of rounds one and two of the BC COVID-19 SPEAK surveys are shown in Table 3.

Table 2 Overall response rate by survey round
Full size table
Table 3 Unweighted and weighted samples of rounds one and two of the BC COVID-19 SPEAK survey
Full size table

In both survey rounds, the response rates were surpassed for the crude provincial target based on sample size calculations and population targets for each of the five HAs (Appendix 1 and Appendix 2). Response rates far exceeded aggregate sample size calculations at a provincial level. However, rural communities, populations with lower educational attainment, lower household incomes, and visible minorities did not meet the HA sample size calculations.

Comparisons of several indicators between the SPEAK and the CCHS samples are shown in Table 4. Self-reported comorbidities of the SPEAK samples for diabetes, heart disease, and cancer (types not specified for each condition) were similar to the 2017/2018 CCHS sample. Self-perceived general health as poor or fair was similar in magnitude across the three BC samples (CCHS and both survey rounds), although slightly higher in the CCHS sample.

Table 4 Comparison of the BC COVID-19 SPEAK survey samples and the Canadian Community Health Survey by key indicators
Full size table

Self-perceived mental health as poor or fair was notably higher at both time points (start of the pandemic and one year later) than the proportion of the BC population who reported poor or fair mental health during 2017/2018. Similar findings for community belonging showed a small weakening at the start of the pandemic compared to the CCHS sample, and this proportion decreased further a year later. The deterioration in mental health and social connectedness are consistent with the current literature relating to the negative impacts arising from the pandemic.

Moderate physical activity of 150 min or greater per week and smoking daily or occasionally were comparable to the CCHS sample.

Key findings

The round one survey showed that, during the early stages of the COVID-19 pandemic, the negative societal impacts were not distributed equitably; the greatest impact was experienced by those with the fewest resources and already experiencing the greatest stress. One year into the pandemic, the second round of the survey showed a further deterioration in health, social and economic impacts, and resiliency, disproportionately affecting those with poorer social determinants of health.

Mental health

There was an increase in the proportion of British Columbians who self-perceived their mental health as poor or fair between survey rounds, and there was a further indication of a decline in mental health with an increase in people who reported worsening mental health (Table 5). There was also an increase in perceived life stress as quite stressful or extremely stressful. Communities across BC reported experiencing a significant increase in reduced connections to family and friends. There was also an increase in people reporting a weak sense of community belonging.

Table 5 BC COVID-19 SPEAK survey — mental health impacts by survey round
Full size table

Young adults

There were significant impacts on young adults aged 18–29 years throughout the pandemic, with substantial disruptions to their mental health, employment, financial security, and life goals (Table 6). Compared to all adults, people aged 18–29 years reported a greater impact on their mental health, with a greater deterioration since the pandemic’s start. They were also twice as likely to report increased difficulty accessing mental healthcare than all adults. A weak sense of connection to their community also increased during the surveys. Current and future financial stress remained high, despite almost three quarters of people reporting they had accessed financial supports or services. Housing and food insecurity remained high between the two surveys for this age group.

Table 6 BC COVID-19 SPEAK survey — young adults (18–29 years) and all adults (18+ years)
Full size table

Households with children

Since the beginning of the pandemic, a greater proportion of households with children reported worsening mental health and financial security than households without children and the negative impacts on their children’s stress and social connections (Table 7). Both types of household compositions reported a weak connection to their community. Parents reported increased stress for children aged 5–17 years throughout the pandemic and reduced social interaction with friends.

Table 7 BC COVID-19 SPEAK survey — households with children
Full size table

Healthcare access

British Columbians reported increased difficulty accessing healthcare since the start of the pandemic, and of those who reported difficulty accessing healthcare, the family doctor, dentist, and diagnostic services were most frequently reported as difficult to access (Table 8). Respondents also reported an increase in avoiding healthcare, with family doctors cited as the most avoided type of healthcare service. In round two, 39.7% of respondents reported their health worsened due to difficulty accessing or avoiding healthcare.

Table 8 BC COVID-19 SPEAK survey — healthcare impacts
Full size table

Vaccine uptake and beliefs

A total of 9.2% reported vaccine hesitancy, with higher levels of vaccine hesitancy reported across some health regions, with Northern HA reporting twice the amount of hesitancy than overall BC, households with an income of less than $20K, individuals with below high school and high school level of education, and people from West Asian or Arab ethnic backgrounds reported higher levels of vaccine hesitancy (Table 9). There was high agreement among respondents in the belief that COVID-19 vaccines were beneficial (89.8%), safe (80.1%), and helpful to get back to everyday life (85.7%).

Table 9 BC COVID-19 SPEAK survey — vaccine hesitancy
Full size table

Adaption

Most British Columbians reported they would like more flexible work options to continue post-pandemic (75.0%); this was highest in people aged 18–50 years (18–29: 80.3%, 30–39: 81.8%, 40–49: 79.2%). The majority (65.2%) of British Columbians also reported wanting to maintain increased access to virtual care; this was highest in people aged 30–69 years (30–39: 69.3%, 40–49: 69.5%, 50–59: 68.8%, 60–69: 65.6%). Most BC respondents would also like to see societal changes to include greater healthcare access (70.9%), reduced income inequality (56.0%), and expansion of green space (54.4%).

Uses of BC COVID-19 SPEAK data

Key indicators were available several months after the closure of the surveys on publicly available dashboards (British Columbia Centre for Disease Control, 2020). Public health decision-makers used key findings to inform policy and prioritize support and public health initiatives to:

  • Inform re-opening plans for safe return to school for kindergarten to grade 12 (Dove et al., 2020) and the return of in-person post-secondary education.

  • Model vaccine projections, inform and target interventions to areas with high rates of vaccine hesitancy, and inform COVID-19 vaccine program decisions and equity considerations.

  • Raise discussions with medical and health leaders around virtual health and healthcare access appropriateness.

  • Raise discussions with community stakeholders to target support and initiatives to improve mental health.

  • Inform recovery priorities in supporting the health and well-being of young adults aged 18–29 years (Samji et al., 2021).

Discussion

The two BC COVID-19 Population Health Surveys represent some of the most extensive known assessments of the societal impacts of the COVID-19 pandemic in Canada and internationally. A consistent and rapid development process at each time point enabled the collection of time-sensitive, population-representative data on many important public health indicators during an evolving pandemic. The short timeline from conception, through development, validation, deployment, analysis, and dissemination, enabled population-specific contemporaneous data to be available to public health decision-makers to inform public health policy, active response, and recovery planning.

The key findings demonstrated both breadth of societal impact and significant inequity in the distribution of negative societal impacts resulting from the pandemic and the public health response early in the pandemic and one year later. The pandemic has disproportionately affected people already experiencing the greatest stresses, notably young adults, families with children, people in lower-income groups, and some ethnic minority groups. The extensive negative impact on physical and mental health, social connectedness, and economic stability as well as resiliency were consistent with the findings of other studies (Douglas et al., 2020; Munasinghe et al., 2020; Statistics Canada, 2018, 2020a; Wang et al., 2020; World Health Organization, 2020). These findings provided insight into the magnitude and distribution of the impact among British Columbians during the pandemic and helped inform several high-priority public health decisions in BC.

Strengths and limitations

There are inherent strengths and limitations to using a cross-sectional study design and a non-randomized quota-based sampling method for rapid population health assessment. Sample sizes were very large for both rounds of the survey; however, some limitations may affect the generalizability of the results and should be considered when interpreting the findings.

A cross-sectional observational study design was used to provide descriptive population data that allowed different population groups and characteristics to be compared at a single point in time. This study design was chosen over other designs, such as a longitudinal study, to provide an informative snapshot of the public disposition quickly and inexpensively in a dynamic and evolving environment; however, due to the inherent limitations of cross-sectional study designs, causal associations cannot be drawn.

A non-probability quota-based sampling method was used to target recruitment for each geographic area for age, sex, income, education, and ethnicity rather than probability random sampling methods. This sampling method was time-efficient and inexpensive to obtain relevant data and meet sample size targets, and the sample size targets for each geographic area were exceeded in both surveys. Despite active and targeted recruitment, some demographic groups and population segments may have been under-represented in the survey. In addition, the option to distribute the survey electronically using a web-based tool may have led to the under-representation of some groups or population segments based on limited internet connectivity, technological proficiency, or geographical location.

Post-collection weighting with Canadian Census estimates by geographic level for age, sex, education, and ethnicity was used to account for the residual differences within the samples and help to minimize bias. The survey data are matched to the 2016 Census estimates. Although the Census may have changed over the last 4 years, the 2016 Census was the best source and most recent to create a population-representative sample. One outcome of the sampling method was that the prevalence of comorbidities (diabetes, heart disease, and cancer) was consistent with those of randomized samples for the BC population reported by CCHS. When comparing several indicators in the SPEAK sample with the CCHS, differences were seen pre-pandemic and throughout the pandemic, which are reflective and consistent with the literature indicating good representativeness of the weighted samples.

Future work

The analysis of the longitudinal subpopulation data to understand the change between the two time points at an individual and population level and qualitative analysis of the open-ended questions will provide an invaluable perspective. Further health assessment of the BC population is needed to understand and address the health, social, and economic impacts and inequities among health outcomes across different population groups. This work will be critical for pandemic recovery.

Conclusion

These population health surveys conducted twice during the COVID-19 pandemic across BC are the most extensive and representative population studies to date. The surveys delivered timely, actionable data to help decision-makers address the burden of the COVID-19 pandemic in BC, informing several critical public health priority activities. As the direct and indirect consequences of the COVID-19 pandemic continue, monitoring and understanding these impacts will be essential over time. This survey methodology provides a rapid and responsive process for population health assessments to inform public health interventions, practices, and policies.

Contributions to knowledge

What does this study add to existing knowledge?

  • The extensive population health surveys consisting of cross-sectional samples of the BC adult population provided insight into the experiences and societal consequences of the COVID-19 response early in the pandemic and one year later.

  • The surveys captured the effect of the pandemic on mental and physical well-being, social connectedness, economic stability, and resilience at provincial, regional, and local levels.

  • Results showed that impacts were extensive and widespread, inequitably distributed, with greater impacts for subpopulations experiencing pre-existing disparities.

  • The survey methodology provides a framework for developing rapid population health assessments to inform and prioritize public health interventions, practices, and policies.

What are the key implications for public health interventions, practice, or policy?

  • The COVID-19 pandemic has exacerbated inequities and existing frailties within healthcare and society, disproportionately affecting some groups, such as young adults, who are not often identified as experiencing health inequities.

  • Rapid large-scale population surveys can contribute to and inform prioritization and targeting public health interventions, practices, and policies and demonstrate the need for ongoing surveillance through short- and longer-term recovery.

  • Consistent methodology used to develop the surveys, grounded within the social determinants of health, provides a framework for developing future population health assessments.