Introduction

The COVID-19 pandemic has triggered a broad range of non-pharmaceutical interventions (NPI), i.e. population-level policies and measures, that aim to prevent and/or control SARS-CoV-2 transmission among individuals and communities [1]. As the pandemic unfolded, more evidence from observational studies emerged, assessing the relative importance and contribution of measures simultaneously implemented.

A systematic review of the methodologies used to assess the effectiveness of NPIs during COVID-19 identified a scarcity of subgroup analysis, necessary to depict differences in effectiveness variation, and also calls for studies where variation in methodologies can be applied, namely through sensitivity analysis that repeat the same analysis with different sets of publicly available NPIs datasets [2]. Other methodological shortcomings challenged by the quality of the outcome data used, which could have been influenced by changes in (surveillance) systems capacities, changing rules or delayed reporting, and that could be at least partly tested by the use of different outcomes within the same study, or testing alternate lag periods for the impact of NPIs [3]. This may help to identify the influence of certain methodological approaches and strengthen the evidence in favour of the effectiveness of particular NPIs.

Previous studies that included European countries found that physical distancing was associated with a reduction in COVID-19 incidence [4]. Restrictions on gatherings, closing of specific sectors (e.g., restaurants, schools, kindergartens, etc.) and closing of some or all school levels were also found to reduce the epidemic growth rare across the 37 Organisation for Economic Co-operation and Development (OECD) members, in an early phase (October-December 2020) [5]. Another study analysing NPI effectiveness across 30 European countries found that a combination of measures involving school closures, banning mass gatherings and early closure of commercial businesses was associated with reduced infections, but other measures, like extensive closure of all non-essential business and stay-at-home orders, were not [6]. These efforts have been confined to the analysis of single outcomes, of single specific measures, single countries, or specific regions, so there is still need to explore the type, combination, or degree of implementation of NPIs that has been effective to mitigate the transmission of SARS-CoV-2 or associated deaths at population level throughout the infection waves. Most data have been made publicly available during the COVID-19 pandemic, which allows testing different methodologies, conducting sensitivity analysis to the effectiveness of the same NPIs, even if differently framed, categorised, and recorded, and seek for commonalities within subgroups of the population (for example, groups of neighbouring, culturally close countries, sharing borders, climate and other contextual features with potential pandemic impact). Outcome data availability also has its shortcomings, since the quality of reporting and recording systems is not assured and bound to changes. Hence, subgroup testing, consideration for varying timing of effects and analytical approaches aiming to rule-out the potential for reverse causation are needed.

In this study, we contribute to the rapidly growing field of evidence on NPI effectiveness, taking some of these methodological concerns into account. We exploit the heterogeneity in timing, temporal sequence, and combination of measures within and across 32 European countries as a natural experiment to assess which combinations of NPIs have been effective to reduce SARS-CoV-2 incidence and associated deaths at the population-level in the early phases of the pandemic. Furthermore, we investigate subgroups of countries (according to geographic region) and apply the same analysis to two different NPIs data sources.

Methods

Study design

The study design resembles a natural experiment [7], in which the populations in countries were repeatedly exposed to different timing and combination of NPIs which ultimately pursued a common aim: reducing population-level transmissions of SARS-CoV-2 and related deaths. Therefore, the effects on outcomes can be studied using each country as their own control, and each country for controls of other countries with different timing or combination of exposures.

Data sources

We resorted to the Corona Virus Pandemic Policy Monitor (COV-PPM) that prospectively monitors and tracks NPIs in 32 countries of the EU27, EEA and UK [8]. A total of eight NPI categories (Panel 1) were retrieved until December 2020, with different sub-categories covering relevant areas of societal living. For each NPI category exact starting dates and duration of implementation are registered in a daily format. A detailed description of methods, data validation process, and usage options can be found elsewhere [8]. The subcategories of retrieved NPIs categories (Panel 1) were combined into categorical variables for analysis as shown in supplementary material (Panel S1).

Panel 1: Categories of non-pharmaceutical interventions (NPI)

a) Restrictions to public events, namely in the number of persons allowed for indoor/outdoor events (conferences, sports, festivals), for more than 1000 persons, less than 1000 persons, more than 50 persons, for any number or without specification;

b) Restrictions or closures to public institutions (incl. schools, universities, public services), in single cities, at the state level, nationally or without specification;

c) Restrictions and closures in public spaces (incl. shops, bars, gyms, restaurants), in single cities, at the state level, nationally or without specification;

d) Measures affecting public transport (incl. trains, buses, trams, metro), in single cities, at the state level, nationally or without specification;

e) Restrictions in movement/mobility, to pedestrians, private cars, national aviation, other means or without specification;

f) Border closures or restrictions applied to travelling by air, land or sea, across national borders, for non-nationals from high-risk regions, for all non-nationals, for all incoming travellers or without specification;

g) Measures relating to Human Resources reinforcement in healthcare (incl. human resources reinforcement or redistribution, technical reinforcement or redistribution, material infrastructural reinforcement or without specification);

h) Masks (mandatory or recommended use of facial and nose protection);

To triangulate measures obtained from other NPI trackers (and allow a posterior sensitivity analysis of models using two distinct exposure measures), we also resorted to the Oxford Covid-19 Government Response Tracker (OxCGRT) [9], and used the specific categories “C1-School closing”, “C2-Workplace closing”, “C3-Cancel public events”, “C4-Restrictions on gatherings”, “C5-Close public transports”, “C7-Restrictions on internal movement”, “C8-International travel controls” and “H6-Facial coverings” (version downloaded on 31.05.2021).

A total of 8512 country-days in 32 countries (see supplementary material Figures S1-8) were analysed stratified by two periods of pandemic waves during 2020 (wave 1: from calendar week 5 to 25, i.e. 27 January to 15 June; wave 2: week 35 to 52, i.e. 24 August to 21 December). The daily number of notified SARS-CoV-2 cases and associated deaths in each country, were retrieved from the WHO [10]. A 7-day smoothed average of reported cases was used as main outcome, to accommodate the expected week-weekend variation in case notification. Country population size (2019) and selected macroeconomic indicators were retrieved from EUROSTAT (https://ec.europa.eu/eurostat/web/main/data/database) for the most recent period available: Gross domestic product (GDP) per capita (Current market prices, million euro, 2019), Health care expenditure (Million Euro per habitant 2017), and Population density (inhabitants per square kilometre, 2018).

Principal component analysis

A principal component analysis (PCA) was conducted to reduce the data collected in the scope of COV-PPM (categories shown in Panel 1) into relevant related factors, reflecting periods when distinct NPIs were simultaneously implemented. Principal component analysis, as a data reduction technique has been regarded as state-of-the-art technique to address the challenge of assessing multiple co-occurring policies in effectiveness studies. The use of PCA, besides aiming to reduce and summarize the number of variables, considers this simultaneous implementation of measures (or is useful “when everything happens at once”) [11]. The goal of PCA is to condense a set of correlated items into a new set of uncorrelated variables, called components. For summarizing correlated NPIs due to their joint implementation, PCA extracts components where the first component explains the maximum variance, and subsequent components explain the remaining variance. Each factor, representing the loadings of individual variables, measures correlations between components and normalized variables, facilitating interpretation in relation to the original variables. Some variables will have stronger correlations with one factor, while others will have weaker correlations across multiple factors.

The scree plots and percentage of variance explained were analysed for each wave to decide the number of components to extract (supplementary material Figure S9). Orthogonal varimax rotation was conducted to identify individual NPIs loadings in each component. Each component score was then scaled by 10 and conversed into scores with non-negative values and a mean of 50.

A descriptive analysis of (means, standard deviations, and within/between country variation over time of NPI categories and derived component scores obtained with COV-PPM are shown in supplementary material, Table S1).

The OxCGRT categories of NPIs “C1-School closing”, “C2-Workplace closing”, “C3-Cancel public events”, “C4-Restrictions on gatherings”, “C5-Close public transports”, “C7-Restrictions on internal movement”, “C8-International travel controls” and “H6-Facial coverings” were entered in a PCA analysis for data reduction, following the same procedure implemented for COV-PPM data. Tables 1 and 2 show the scoring coefficients of the resulting components obtained using each NPIs database, respectively. By using PCA applied to both policy trackers also enables an easier (side-by-side) comparison between NPIs implemented during this period.

Table 1 Scoring coefficients for orthogonal varimax rotation, non-pharmaceutical interventions (Corona Virus Pandemic Policy monitor - COV-PPM)
Full size table
Table 2 Scoring coefficients for orthogonal varimax rotation, non-pharmaceutical interventions (Oxford Covid-19 Government Response Tracker - OxCGRT)
Full size table

Three components per wave, i.e. sets of combinations of NPIs, explained 74% of the variance for the first (C1-3) wave and 70% of the variation in the second wave (C4-6) for COV-PPM categories.

In the first wave, C1 was strongly related with NPIs referring to restrictions in movement/mobility (scoring coefficient for orthogonal varimax rotation: 0.6512), and, to a lesser extent, related with measures affecting public transport (0.4371), public events (0.3660) and public spaces (0.3616). C2 was mainly related with measures that aimed at improving the healthcare system (0.7349), border closure/travelling restrictions (0.4549) and measures impacting public institutions functioning (0.3208). C3 was related with recommendations or enforcement of mask utilization (0.9533).

In the second wave, C4 was related with restrictions in public events (0.5822), in public spaces (0.5327), border closures/travelling restrictions (0.4218) and public institutions (0.3271). C5, was related to restrictions to movement/mobility (0.7520), public transport (0.4868) and healthcare system improvement measures (0.3126). C6, was also primarily related with masks (0.8465).

For the OxCGRT, in wave 1, C1-Ox was related to measures on international travel control (0.5901), public events restrictions (0.4741), restrictions in gatherings (0.3601), school closing (0.3827) and workplace closing (0.3531). C2-Ox, was related to restrictions on public transport (0.7835) and internal movement (0.4520). C3-Ox was related to facial coverings (0.9839). In wave 2, C4-Ox was mainly related to restrictions in gatherings (0.4697), facial coverings (0.4339), workplace closing (0.4335), cancelation of public events (0.4155), and internal movement restrictions (0.3678). C5-Ox was primarily related to international travel control (0.7167) and C6-Ox was related to public transport related measures (0.7948).

Statistical analysis

Scatter plots were used to explore the temporal change in NPIs as measured with COV-PPM and SARS-CoV-2 incidence (per 100,000) by country (supplementary material, Figures S1 – S8). For descriptive purposes, the proportion of observation time in which COV-PPM NPIs were in place, and the within and between country variation therein, was calculated by country and wave (supplementary material, Table S1).

A panel analysis was implemented to analyse the effect of NPIs on daily SARS-CoV-2 incidence and associated deaths by calculating incidence rate ratios (IRR) and corresponding 95% confidence intervals (CI). The IRR estimates hence quantify the relative difference in SARS-CoV-2 incidence or associated deaths for a one-unit change in component scores over time (a) between countries, comparing countries with different NPI component scores, and (b) within countries, comparing variation in respective NPIs implemented and their stringency (simultaneous presence of categories of each measure).

Different model specifications fitted to the data were tested (Supplementary material, Table S2 and description). A multi-level mixed-effects negative binomial model with country as random-intercept showed the best model fit according to lowest Akaike (AIC) and Bayesian Information Criteria (BIC) (supplementary material, Table S2).

The analysis was guided by a causal diagram (Fig. 1), illustrating the potential causal/non-causal pathways (dashed lines) from NPIs implementation to outcomes over time. Crude models were fitted including the components obtained by PCA, illustrated as C1-C3 for wave 1 and C4-C6 for wave 2 and the outcome (left section of the graph, with dashed lines linking to Incidence at T1 and showing the “immediate” or baseline effects). Models were further adjusted for one of five time-lagged variables of the same components in five separate models, respectively (crude model, and models adjusted for 7-, 14-, 21-, 28-, and 35-days lag of each component, respectively). The crude models (no lag) show the immediate association with the outcome of each component, adjusted for the effect of one another. In other words, we added the term with no-lag in the models to adjust for the presence of a “baseline” or crude effect, that would allow to measure the independent effect of each added variable, i.e., variables with the effect of measures after 7-, 14-, 21-, 28, and 35- days, adjusted for the “immediate” association present at NPI implementation. The IRR estimates obtained from the further adjusted models show the independent lagged association of each component (represented in Fig. 1 as the straight lines from Components C1-C6 up to their removal, horizontally, on the right side), i.e. adjusted for the non-lagged association of each component with the outcome (incidence or deaths) and the lagged association of the other components (represented in the right section of Fig. 1, as straight lines from Components C1-C6 to Incidence at Tx). The lags considered for deaths were 21-, 28-, 35-, 42-, and 49-days. The choice for 7-days increasing lagged-association was based on SARS-CoV-2 natural history and the existing literature [12, 13].

Fig. 1
figure 1

Causal diagram

Full size image

All models further included time (in days) as discrete variable to consider secular trends by unobserved variables (or any time-varying aspect unrelated to NPIs), the population size (as offset), and country (as random intercept). NPIs have often been implemented reactively following a rapid increase in SARS-CoV-2 cases, and have been lifted, likewise, following a rapid decline in incidence raising issues of reverse causation when studying their impact on infection dynamics. Therefore, a variable for the change in the rate of SARS-CoV-2 incidence from 7 days before (see calculation below) was entered in all models to account for potential reverse causation, i.e. preceding increase or decrease in incidence rates impacting the timing and stringency of NPIs implementation.

The mixed-effects negative binomial models with \({Y}_{it} \sim \,NB({\overline{y}}_{it},\:d)\), where \({Y}_{it}\) are the observed and \({\overline{y}}_{it}\) the corresponding mean number of cases or death Covid-19 cases or deaths, and \(d\) the dispersion parameter.

For country \(i=\left\{1,\:\dots\:,\:32\right\}\), and day \(t=\{1,\:\dots\:,\:280\}\) the rate \({\eta}_{it}={log}\left(\frac{{\overline{Y}}_{it}}{{P}_{i}}\right)\) of average number of cases or deaths \({\overline{y}}_{it}\), and population size \({P}_{i}\) used as an offset \(\text{log}\left({P}_{i}\right)\) was specified on a logarithmic scale as:

$${\eta}_{it}={\beta}_0+{\beta}_1{day}_t+\sum\limits_{y=2,x=k}^{y=4,x=n}{\beta}_yC_{xit}+\sum\limits_{y=5,x=k}^{y=7,x=n}{\beta}_y{C\_lagged}_{xi(t-j)}+{\beta}_8{Rate\_change}_{it}+{\gamma}_i$$

where \({\beta}_{0}\) is the intercept, \({\beta}_{1}\) is the coefficient of the identifier variable for days \({day}_{t}\), \({\beta}_{2}\) to \({\beta}_{4}\) are the coefficients for the three component scores \({C}_{xit}\) of each wave, where component identifier \(x=\{1,\:2,\:3\}\) used for modelling the first wave (i.e. \({C}_{1it}\), \({C}_{2it}\) and \({C}_{3it}\)), and \(x=\{4,\:5,\:6\}\) used for modelling the second wave (i.e. \({C}_{4it}\), \({C}_{5it}\) and \({C}_{6it}\)). \(k\) and \(n\) are the lower and upper limit of summation for the respective value set of \(x\). \({\beta}_{5}\) to \({\beta}_{7}\) are the coefficients for the temporal lagged component scores \({C\_lagged}_{xi(t-j)}\) using different component identifier \(x\) for the first and second wave (see above), while lagging the respective component score by j-days (for cases \(\:j=\left\{7,\:14,\:21,\:28,\:35\right\}\), and for deaths \(\:j=\left\{21,\:28,\:35,\:42,\:49\right\}\)), which were entered in separate models for each level of j. In other words, \({\beta}_{5}\) to \({\beta}_{7}\) can be thought to represent the trend in incidence after (j-days of) implementation of the respective NPIs.

\(\:{\beta\:}_{8}\) is the coefficient for the change in rate \({Rate\_change}_{it}\) in Covid-19 incidence 7 days before, i.e. \({Rate\_change}_{it}={Y}_{it}/{Y}_{it-7}\) (entered in models for cases and deaths). \({\gamma}_{i}\) is the random effect modelled using exchangeability among countries, and \(\:{log}\left({P}_{i}\right)\) is the country population size entered as offset.

GDP, healthcare expenditure per capita, and population density were further tested as covariates, but not included in the final models since they did not improve model fit nor change the magnitude or direction of considered effects (supplementary Tables S12 and S13).

The final models were additionally stratified according to geographical regions in Europe: Southern (Portugal, Spain, Italy, Greece and Cyprus), Western (Belgium, Netherlands, France, Germany, Ireland, United Kingdom and Austria), Eastern (Czech Republic, Slovakia, Slovenia, Poland, Romania, Hungary and Bulgaria), Northern (Norway, Sweden, Finland and Denmark) and Other regions (Croatia, Estonia, Iceland, Latvia, Lithuania, Luxembourg, Malta, Switzerland and Liechtenstein).

All analysis were conducted using Stata 16® [14], visualisations were performed with R programming language 4.1.2. Geographic data for maps were retrieved from Eurostat [15].

Results

Descriptive analysis

Across all countries, a total of 1,614,594 COVID-19 cases and 178,369 associated deaths were analysed during the first wave and 18,471,042 cases and 328,426 deaths during the second wave (Fig. 2). The timing of NPI implementation and the proportion of days during the observation period in which measures were in place across countries varied widely within and between countries during the two infection waves. In wave 1, restrictions to public events, public institutions, and public spaces were the most widely used measures (Fig. 2, and supplementary material Table S1). In wave 2, these measures were partly relaxed, while mask policies, restrictions to travelling and border closures, and movement/mobility restriction became more prominent (Fig. 2). Within and between country variation in NPIs remained high with the exception of masks where variation (especially within countries) was lower (supplementary material, Table S1).

Fig. 2
figure 2

Proportion of days during the observation period in which NPIs were in place (Wave 1 - weeks 5–25, and Wave 2 – weeks 35–52), and cumulative incidence and mortality rates per 100,000 of SARS-CoV-2, in 32 countries (EU-27, EEA, UK). Legend: (1), (2), (3) and (4), refer to the simultaneous presence of sub-categories in place for each non-pharmaceutical intervention (i.e., 1: if at least one of the subcategories was present, 2: if at least two of the subcategories were simultaneously present, 3: if 3 subcategories were simultaneously present, and 4: if 4 subcategories were simultaneously present)

Full size image

NPI effects on SARS-CoV-2 incidence

Incidence rate ratios (IRR) for the lagged-effects of the PCA-components of NPIs derived from COV-PPM, considering a 7, 14, 21, 28 and 35 days-lag across all countries and stratified by waves and regions are presented in Fig. 3.

Considering the findings across all countries during the first wave, the NPI combination C1 (movement/mobility, public transport, public events, public spaces) was significantly associated with a reduction in SARS-CoV-2 incidence in models considering a 28-days and 35-days lagged effect (adjusted for the baseline effect): IRR, 95%CI = 0.995, 0.992–0.999 for the 28-day-lagged variable, 0.994, 0.990–0.997 for 35-day-lag). C2 (healthcare system improvement, border closures and restrictions in public institutions) revealed the same pattern as C1 across all countries, with significant association with lower incidence, IRR after 21-days (0.993, 0.987–0.998), 28-days (0.987, 0.982–0.992) and 35-days (0.984, 0.979–0.988).

C3 (masks) was associated with lower incidence in the non-lagged model (IRR = 0.962, 0.955–0.969) and for all the time-lags considered except for a 35-days lag.

In southern countries during the first wave (Fig. 3), C1 was associated with a reduction in incidence after 28-days, while C2 was associated with higher incidence after 7, 14, 21 and 28 days-lag. C3 was associated with lower SARS-CoV-2 incidence for all time-lags considered except after a 28 and 35-days lag.

In Western countries, C1 and C2 were associated with lower incidence after 28 and 35 days, while C3 was significantly associated with a reduction in incidence for all time-lags considered.

In the Eastern region, C1 and C3 were significantly associated with a reduction in incidence rates for all lags considered (except C1 for “no-lag” effect), while C2 was associated with an increase in incidence for all lagged variables.

In Northern countries, C1 showed associations with an increase in incidence rates for 14, 21 and 28-days lag, while C2 and C3 were significantly associated with a decrease in incidence for all lags considered (except C2 after 7 days and after 35 days-lag, where no significant association was observed).

In the remaining countries analysed as residual group (Croatia, Estonia, Iceland, Latvia, Lithuania, Luxembourg, Malta, Switzerland and Liechtenstein), C1 and C2 were significantly associated with lower incidence after 28 and 35 days while C3 was significantly associated with lower incidence after 14, 21 and 28 days.

Results of all full models estimates (i.e., including estimates for non-lagged effects for all models and corresponding 95% confidence intervals, CI) are shown in supplementary material (Tables S3, S4 for all countries and Table S8, S9, stratified by geographical regions and S12, S13 adjusted for GDP, healthcare expenditure and population density).

Fig. 3
figure 3

Incidence Rate Ratios (IRR) and 95% CI for the lagged-effects (7, 14, 21, 28 and 35 days) on SARS-CoV2 incidence of the three principal component (PCA) scores of NPIs, N = 20,085,636 SARS-CoV-2 cases in 32 countries during the first and second waves of infections (March-December 2020), by region. All countries: includes all countries of EU27, EEA and UK; Southern: Portugal, Spain, Italy, Greece and Cyprus; Western: Belgium, Netherlands, France, Germany, Ireland, United Kingdom and Austria; Eastern: Czech Republic, Slovakia, Slovenia, Poland, Romania, Hungary and Bulgaria; Northern: Norway, Sweden, Finland and Denmark; Other: Croatia, Estonia, Iceland, Latvia, Lithuania, Luxembourg, Malta, Switzerland and Liechtenstein. Within each regional stratum and column for time lags, IRR estimates represent the independent effects of C1- C3 (wave 1) and C4-C6 (wave 2), i.e. mutually adjusted for one another, and additionally adjusted for the immediate effect of each component (no lag), time, 7-day rate change in incidence, and country as random intercept

Full size image

In the second wave (Fig. 3), restrictions in public events, public spaces, border closures and restrictions in public institutions (C4) had no significant association with incidence for any of the time-lags considered in all countries (except for the association with an increase in incidence in the non-lagged variable).

Restrictions in movement/mobility, public transport and healthcare system improvement measures (C5) were significantly associated with lower incidence following 21-days (0.990, 0.986–0.994) 28-days (0.980, 0.976–0.984) and 35-days (0.975, 0.971–0.979). C6 (positively related with masks, but, although showing a weaker relationship, negatively related to border closure/travelling restrictions and healthcare system improvements) unfolded significant associations with an increase in incidence for all time-lags considered, with the point estimate showing a decrease in magnitude with increasing lagged days.

In the Southern region, C4 was significantly associated with lower incidence in the non-lagged model and after 7- and 14-days, but showed no significant associations for the remaining lags considered. C5 and C6 was significantly associated with lower incidence after 21 (for C5), 28 and 35-days.

In the Western region, C4 measures were associated with an increase in incidence when considering all time-lags except the 21-days. C5 was significantly associated with lower incidence for all the time lags considered while C6 showed no significant association with incidence for all the time lags considered.

In the Eastern region, the components C4-C6 showed a trend of associations suggesting lower incidence according to the increase in time-lag considered, with significant associations with a reduction in incidence for C6 after 14, 21, 28 and 35 days and for C4 and C5 after 28 and 35 days.

In the Northern region, C4 and C5 were significantly associated with lower incidence after 21, 28 and 35 days, and C6 showed an association with an increase in incidence for all the time-lags considered.

In the “other” regions, C4 and C5 showed no significant associations on SARS-CoV-2 incidence (except for C4 after 14 days associated with lower incidence), while C6 showed a significantly association with higher incidence for all time-lags analysed.

Associated deaths

Considering all countries in the first wave (Fig. 4), C1 was associated with an increase in deaths after a 21-,28-, and 35-days lagged association (IRR, 95%CI for no lag = 1.124, 1.119–1.129, for 21-days lag = 1.037, 1.033–1.040, for 28-days = 1.019, 1.015–1.022, for 35-days-lag = 1.008, 1.004–1.011), which then turned into an association with a decrease in deaths after 49-days (0.994, 0.991–0.997). C2 showed the same trend as C1, associated with an increase in deaths for no-lag (1.060, 1.053–1.067), 21-days (1.014, 1.009–1.018), 28 days (1.007, 1.002–1.012) and with a decrease after 49-days (0.990, 0.986–0.995). C3 was significantly associated with lower deaths for all the time lags considered across all countries, except after 42 and 49 days, when estimates were non-significant.

Fig. 4
figure 4

Incidence Rate Ratios (IRR) for the lagged-effects (21, 28, 35, 42 and 49 days) on incident SARS-CoV-2 associated deaths of the three principal component (PCA) scores of NPIs, N = 506,795 deaths in 32 countries during the first and second waves of infections (March-December 2020), by region. All countries: includes all countries; Southern: Portugal, Spain, Italy, Greece and Cyprus; Western: Belgium, Netherlands, France, Germany, Ireland, United Kingdom and Austria; Eastern: Czech Republic, Slovakia, Slovenia, Poland, Romania, Hungary and Bulgaria; Northern: Norway, Sweden, Finland and Denmark; Other: Croatia, Estonia, Iceland, Latvia, Lithuania, Luxembourg, Malta, Switzerland and Liechtenstein. Within each regional stratum and column for time lags, IRR estimates represent the independent effects of C1- C3 (wave 1) and C4- C6 (wave 2), i.e. mutually adjusted for one another, and additionally adjusted for the immediate effect of each component (no lag), time, 7 day rate change in incidence, and country as random intercept

Full size image

In the Southern region during the first wave, a significant (p < 0.01) association with a reduction in deaths was only noted for C1 after a 49-days lag, and the same C1 was associated with an increase in the non-lagged variables and after 21 and 35 days. C2 also showed significant associations with an increase in deaths after 21 and 28 days. C3 was significantly associated with lower deaths in the non-lagged model and in models lagged at 21, 42 and 49 days.

In the Western region, C1 showed a significant association with an increase in deaths in non-lagged model and was significantly associated with lower deaths after 28, 35, 42 and 49 days, while C2 was significantly associated with lower deaths after 28, 35 and 42 days. C3 was significantly associated with a reduction in deaths for all time lags considered.

In the Eastern region, C1 was associated with an increase in deaths in the “no-lag” model, but with a reduction in deaths for all the time-lags considered. C2 was associated with an increase in deaths for all time-lags (including the non-lagged model). C3 was associated with a reduction in incident deaths after 21, 28, 35 and 42 days.

In the Northern region, C1 was associated with an increased deaths in the non-lagged model and after 21 and 28 days. C2 was associated with an increase in deaths in the non-lagged model and was significantly associated with a reduction in deaths after 21 and 28 days. C3 was significantly associated with lower incident deaths in the non-lagged model and after 21, 28 and 35 days lag, but was associated with an increase in deaths after 42 and 49 days (although with large confidence-intervals for the latter two estimates).

In the “other” countries, C1 and C2 showed initially significant associations with increased number of deaths (no-lag). C1 unfolded significant associations with reduced number of deaths after 28, 35 and 42 days, while C2 unfolded significant associations with reduced number of deaths for all the time-lags considered. C3 was significantly associated with a reduction in incident deaths after 21 and 28 days, but turned into significantly associated with an increase in deaths after 42 and 49 days.

During the second wave in all countries (Fig. 4), C4 was significantly associated with a reduction in deaths only in the non-lagged model (IRR, 95%CI = 0.994, 0.991–0.998), and showed a significant association with increased number of deaths after 28 days (1.004, 1.001–1.007). C5 was significantly associated with a reduction in incident deaths after 28, 35, 42 and 49 days (having started associated with increased deaths in the non-lagged model). C6 showed significant association with increased deaths for all lags considered.

In Southern countries, C4 was significantly associated with a reduction in deaths in the non-lagged model and after 42 and 49 days. C5 was associated with increased number of deaths in the non-lagged model, but was significantly associated with a reduced number of deaths for all the time-lags considered. C6 showed an association with increase, and turned into a significant association with a decrease after 28, 35, 42 and 49 days.

In the Western region, C4 showed significant associations with an increase in deaths for all time-lags except after 21, 42 and 49 days. C5 was associated with an increase in deaths in the non-lagged model, but was significantly associated with a reduction in incident deaths for all time-lags considered thereafter, while C6 did not reveal any significant associations on deaths in this groups of countries.

In the Eastern region, C4 was significantly associated with a reduction in deaths following 28, 35, 42 and 49 days, while C5 was significantly associated with a reduction in the number of deaths after 28 and 35 days. C6 was significantly associated with lower deaths for all time-lags.

In the Northern region, C4 was significantly associated with lower deaths for all time-lags analysed. C5 was also significantly associated with lower deaths for all time-lags except after 42 days (non-significant) and 49 days. C6 was associated with an increase in deaths for all time-lags (no-lag, 21, 28 and 35-days) that turned into a significant association with a decrease in deaths after 49 days.

In the group of “Other” countries analysed, C4 did not show any significant association, while C5 was significantly associated with an increase in deaths in the non-lagged model and after 49 days. C6 showed an association with an increase in deaths for all the time-lags considered.

Supplementary tables S5, S6 show all estimates obtained from models fitted for all countries, tables S10, S11 show these stratified by country groups and tables.

Sensitivity analysis

The analysis using NPIs collected by OxCGRT showed similar results to those obtained with NPIs collected by COV-PPM for all countries and in the analysis stratified by region, for cases and deaths, Figs. 5 and 6 (supplementary Figure S10 and S11 show IRR estimates and 95% CI).

During the first wave, C1-Ox and C2-Ox, were positively associated with incident cases of SARS-CoV-2 when considering their “non-lagged” associations, and then showed significant negative associations after 21-days, across all countries (supplementary Figure S10). C3-Ox revealed a significant negative association during the first wave for the non-lagged association, which turned into a significant positive association following 21, 28 and 35 days. During the second wave, C4-Ox revealed a significant positive direct (i.e., non-lagged) association, that turned into a significant negative association for all time-lags analysed, again across all countries, and C5-Ox was significantly negatively associated with cases for all time-lags considered. During the second wave, C6-Ox was initially negatively associated with incident cases (no-lag) and also after 35 days, across all countries.

The stratification according to regions obtained using OxCGRT components, in general suggested similar patterns as those obtained with the components summarised using COV-PPM’ group of measures, as shown in Fig. 5.

Fig. 5
figure 5

Heat map comparing estimates obtained with COV-PPM and OxCGRT for models predicting incident cases of SARS-CoV-2 (i.e., Fig. 3 and Figure S16 – only colour codes)

Full size image

Regarding deaths (supplementary Figure S11) and across all countries, C1-Ox and C2-Ox were positively associated during the first wave considering a “non-lagged” association, while C3-Ox showed a negative “non-lagged” association. During this wave, C2-Ox turned to a negative association following 21, 28 and 35 days, while C1-Ox showed positive associations for 21, 28 and 35 days, that unfolded to significantly negative association after 42 and 49 days. C3-Ox showed significant positive associations after 21, 28 and 35 days.

During the second wave, C4-Ox and C6-Ox showed a positive “non-lagged” association that became negative after 21 days (C4-Ox) and 35 days (C6-Ox) and remained significant for the following time-lags analysed, across all countries. C5-Ox was negatively associated with deaths across all countries for all the time-lags analysed. Again, the regional stratification showed patterns of associations similar to those obtained with COV-PPM, shown in Fig. 6).

Fig. 6
figure 6

Heat map comparing estimates obtained with COV-PPM and OxCGRT for models predicting deaths of SARS-CoV-2 (i.e. Fig. 4 and Figure S17 – only colour codes)

Full size image

Discussion

In this study we analysed the impact of NPIs, prospectively recorded across 32 European countries, on SARS-CoV-2 incidence and deaths during two pandemic waves of 2020 using principal components derived from two different NPI trackers (COV-PPM and OxCGRT) as exposures. During the first wave of infection, the three component factors (summarizing the effects for all the NPIs analysed) were associated with a reduction in the incidence of SARS-CoV-2 across all countries, albeit with varying time lags. In the second wave, only restrictions to movement/mobility, public transport and healthcare system improvements (C2) were significantly associated with a reduction in incident cases across all countries.

Regional stratification allowed to differentiate the patterns of these associations for each wave, and showed, for example, that the component C3 related to “masks” was consistently associated with lower incidence of SARS-CoV-2 cases in all regions during the first wave. However, such associations with reduced incidence were only noted in the Southern and Eastern regions during the second wave, while an inverse association was found in all other regions for at least one of the time-lags considered (adjusted for all other NPI effects).

For deaths, our results suggest that most measures were associated with a time-lagged decrease in mortality. An exception was C3 (masks) during the second wave, which showed an association with increased number of deaths when considering all countries. Only in Southern and Eastern regions (and Northern after 49 days) were masks significantly associated with lower deaths, showing again the importance of regional variation in NPI impacts.

We obtained consistent results when analysing the associations between NPIs and SARS-CoV-2 cases and deaths using the OxCGRT NPI tracker as exposure (compared to using COV-PPM), with some exceptions. NPIs were associated with a reduction in cases and deaths, and the exceptions to this pattern observed according to geographical regions, were congruent when comparing both NPIs trackers: for example, during wave 1, COV-PPM’ C2 (healthcare system improvements, border closures) and OxCGRT’ C1-Ox (international travel control, public events, schools, workplace restrictions) did not suggest an association consistent with a reduction in cases in the Eastern region, and also did not suggest a reduction in deaths in the same region (except C1-Ox, but only after 49 days). The same was seen for C6 (masks) and C5-Ox (facial coverings), but also for C6-Ox (public transport measures), which were not associated with a reduction in cases in the Northern region during wave 2. For C3 (masks) and C3-Ox (facial coverings) during wave 1 and across all countries, an association with a reduction in cases was initially observed, which disappeared after 35 days using COV-PPM data and which turned positive after 28 and 35 days with OxCGRT, and a very close pattern was noticed for deaths during this wave. However, during the second wave, across all countries, only C5-Ox (facial coverings), did suggest an association with a reduction in cases and deaths.

Overall, the negative associations observed between NPIs on SARS-CoV-2 incidence (i.e., the associations with a reduction in incidence, expressed as IRR below the unit) suggests that the pandemic control strategies were linked with a reduction, at least partially, of the incidence across these countries at population-level. It should be noted that our observations of “what might have worked” may only be valid for the observed periods in 2020, i.e., where the population was immunologically naive, had no access to vaccination and was faced with a respiratory virus with a comparable natural course of diseases or similar reproduction rate. With the emergence and roll-out of COVID-19 vaccines, the suggested inference of our effectiveness estimates beyond this time period may be limited. However, in case of escape variants that evade immunity acquired through vaccines or previous infections, the knowledge generated from this study could inform the design and combination of future NPIs to protect immunologically naive populations against the threat of respiratory virus.

Our results concur with and add to previous observations of the effect of physical distance interventions imposed across 149 countries, which were associated with a reduction in COVID-19 incidence [4].

The recommended or compulsory use of masks seemed to be consistently associated with a reduction in SARS-CoV-2 cases and deaths at the population-level according to our findings, with such association during the first wave of infection being noticeable for all the time-lagged associations considered and across all countries. This is in line with a review showing the potential effectiveness of mask utilization in community settings, particularly in the case of specific mask types (i.e. medical types), [16] with a recent review on the effectiveness of NPIs to reduce SARS-CoV-2 transmission [17] and with a cluster-randomized trial conducted in Bangladesh that showed the community-level effect of masks distribution on SARS-CoV-2 cases [18]. However, a re-analysis of this trial data showed that its results were not free of relevant bias (unblinding, ascertainment bias, and bias-susceptible endpoints), which made its results consistent only with a modest or having no direct effect on COVID related outcomes [19]. Also, an updated Cochrane review on physical interventions to interrupt or reduce the spread of respiratory viruses (including mostly studies from previous epidemics or pandemics), did not conclude for a reduction in respiratory viral infection with the use of medical or surgical masks [20].

The fact that “masks” were not consistently associated with a reduction in the number of SARS-CoV-2 cases during the second wave across all countries may be due to the homogeneity of this measure during this period, with almost all countries (except Northern countries) recommending the use of facial masks in that time period so that variance was substantially reduced and proper counterfactuals were lacking in the second wave to detect the effect of the measure. Notwithstanding, we decided to show such associations to reinforce the congruency of results, i.e. that an effect would not be so apparent under such homogenous circumstances, as was the case.

The stratified analysis by waves also revealed different magnitudes of associations for the same components, i.e. the same set of combinations of NPIs, included when comparing waves. These differences may be due to organizational safety measures and individual protective behaviours in place already during the first wave, thus reducing the additional effect of newer implementations, as previously suggested [21]. Again, this can also help explain the seemingly strong association observed for recommended or mandatory use of “masks” in the Northern region during the first wave, reflecting the late and looser approach taken to NPIs implementation in some of the countries in this region.

Other measures, such as border closures or travelling restrictions, as part of other NPIs, seemed associated with a reduction incidence, particularly in the first wave, but not in all regions. Their effectiveness may strongly depend on the timing of implementation, which is also in line with the results of a Cochrane review that included mainly modelling studies suggesting that such measures may lead to a reduction in the number of new cases, although with large uncertainty and when implemented at the beginning of the outbreak [22]. Travel restrictions were also analysed using the ECDC categorization of NPIs for the European region [1], showing different impacts, dependent on the starting date and on the combination of NPIs in which they have been implemented [23] (e.g., quarantine of incoming travellers, enforcement of hygiene concepts and limitations to and from specified high-risk regions).

Across all waves and regions, the analysed NPI components also showed non-significant or significant associations with higher incidences or deaths. These non-significant associations may suggest that the respective measures had no measurable effect at the population level, when adjusted for the immediate effect and the co-occurrence of other NPI combinations. However, this does not allow the inference that a given measure is completely ineffective at a smaller scale (e.g. in a given region or town of a country) or at the individual-level. Although we adjusted for the 7-day preceding change rate in incidence, reverse causation for the immediate (non-lagged) effects that show associations with higher incidences and deaths cannot be completely ruled out as it is more likely that measures were taken because of high infections rates or deaths, and not that measures caused higher infections and deaths. In the lagged models, positive associations (i.e. IRR above the unit, or associated with an increase) on incidence or deaths may mean that the measures were not able (e.g. possibly due to improper design, implementation, stringency, or adherence) to translate into a measurable change in pre-existing rising trends of infections or deaths, which were the likely reason for measures’ implementation. The results from models fitted only with the lagged effects on incidence across all countries (without the immediate effect variables, results not shown), support our interpretation by showing the same pattern of associations. However, we cannot rule out residual confounding by other unmeasured variables that may well lie in the causal chain explaining the association between NPIs and the outcomes [24]. Nevertheless, the use of two different exposures (despite also the uncertainty about the delay between NPIs implementation), constitutes an attempt to reduce the potential bias stemming from the use of different outcomes (e.g. incomplete cases, different testing capacities), which has been rarely tested in other studies [2].

Further explorations, e.g. through structural equation models, are needed to more clearly elucidate the causal pathways related to NPIs and pandemic control, such as the role of the weather, seasonality, or media communication, political discourses, individual behaviour, social support networks, or trust.

Strengths and limitations

We used a standardized procedure to monitor and code NPIs taken across Europe and resorted to a natural experiment approach [25], which stands as a suitable and strong research design to monitor the measures taken to control the pandemic. Implemented in a panel design, each country serves as its own control while cross-national differences are considered simultaneously. Our estimates are, however, limited by the fact that they do not disentangle the effect of each NPI individually, but assessed the independent effect of three sets of NPI combinations that explained a high proportion of the variance across Europe (although each individual NPIs has different loadings in each component, as shown in Table 1). In other words, the use of principal component analysis to summarize data means that we can only refer to the effect of the group of related measures in each component, limiting the attribution of direct effects to specific categories of NPIs. For example, C1 corresponds to periods when packages of NPIs have been implemented, with stronger emphasis on movement/mobility, public transport, public events, public spaces, and less contribution of the other NPIs (although, they were still present, but with a lower contribution to the summary of the variance represented by the component). If C1 and C2 both associate with a decrease in incidence, it indicates that the correlated variables summarized in each component relate to the change in incidence. While we cannot isolate the specific effect of individual measures like mobility restrictions, we can state that such measures relate more to a higher score in Component 1. If this component is negatively associated with incidence decrease, it implies that the variance summary of the related variables is linked to this change in incidence. Conversely, the opposite applies for an increase in incidence. Despite this limitation, a principal component analysis is regarded state of the art to address the challenge of multiple co-occurring policies in effectiveness studies [11]. Furthermore, NPIs were implemented as “packages” of combined measures in all countries, which hinders the possibility of analysing their individual association with n incidence rates. Future studies should analyse the independent effects of individual NPIs, and also explore the use of other outcome measures such as the reproduction number.

We further cannot rule-out the possibility of misclassification of NPIs, particularly within some of the subcategories of the measures established. Furthermore, we did not account for the heterogeneity in the baseline status of, for e.g., healthcare infrastructure, that could determine a larger or smaller association for any particular measure to improve the healthcare system response. However, the congruency in the association of some of the NPIs summarized in the stratified analysis by regions, is in favour of an independent effect across different constellations of systems, despite the needed (and legitimate) simplification by attributing the same meaning to each group of NPIs for each of the 32 countries.

Other structured data collection efforts are being developed by different consortia, categorizing NPIs across the globe using different criteria and aggregation levels and updated with different periodicity [26,27,28,29], allowing to further assess NPIs effectiveness. We conducted a sensitivity analysis with selected NPIs covered in the OxCGRT [9], and obtained patterns that confirm our findings, despite the inevitable differences in NPIs categorisations.

The use of notified cases and deaths might be a further limitation of this study, due to potential changes in national notification processes or testing intensity, for example, which can directly impact these outcomes. However, the use of different lag periods to measure the associations between NPIs and different outcomes (including the analysis with “no-lags” provided as supplement), and the congruency in the results obtained using the two different exposures, can also be seen as a sensitivity analysis to the potential bias imposed by outcome data quality, as previously suggested by authors calling for rigorous impact analysis in this field [3]. Nonetheless, further testing should be done in subsequent studies, to rule-out any impact of changes in surveillance system, case ascertainment, or cause of death attribution rules, for example.

An analysis conducted with the “Stringency index” of NPIs proposed in the scope of the OxCGRT, showed that the timing of restrictive measures implementations seems crucial to mitigate SARS-CoV-2 incidence [30], but strategies did not have the same effect in all the countries with available data. Another analysis relating the “Stringency Index” aggregated at the continental level, with COVID-19 Case-fatality Rates (CFR) worldwide, did not observe a statistically significant association between the Index and COVID-19 CFR [31]. The authors found that stricter measures were associated with higher CFR in high-income countries with active testing policies (testing anyone symptomatic or testing open to public), suggesting that more restrictive (lockdown) measures might hit the most vulnerable groups harder [31]. Other research also found that more stringency in NPIs led to fewer deaths [32]. Such results obtained with the OxCGRT also suggest the need for further research that considers socioeconomic (and inequality) aspects of the measures taken for pandemic control.

As NPIs are implemented and withdrawn dynamically, attempts to track these interventions need to embrace a continuous effort within an appropriate monitoring framework [21]. A modelling effort conducted for 16 different countries, suggest that implementation of dynamic interventions (i.e., alternating between periods of NPIs enforcement followed by periods of relaxation), might not be ideal [33]. In another analysis, the combination of physical distancing measures implemented with varying intensity and timing (border controls, restriction on mass gatherings, lockdown type measures) seem to be effective if implemented early (about two weeks before the 100th case), although individual effect is hard to disentangle since several measures were implemented very close to one another [34].

Changes in NPIs effectiveness across time are expected since their stringency changes (e.g. from recommendations to impositions with envisaged punishment for noncompliance), measures are refined (e.g., continuous adaptation of infrastructures to ensure social distance, new air ventilators/purifiers) and the population also adapts to and evolves in the way measures are understood and followed. The effectiveness may hence be very time- and context-dependent. This adds to the complexity in the understanding of NPIs’ effectiveness, and substantiates the research challenges ahead to provide relevant information for public health decision-making.