Introduction

The COVID-19 pandemic has had a major impact on public health, the healthcare system, and our daily lives. To avoid rapid spread of the virus, most countries imposed numerous nationwide measures, including social distancing requirements, closing of schools, and severe restrictions regarding socializing, exercise, and leaving the home. Although adolescents are generally not at risk of developing severe COVID-19 [1, 2], the effects of a lockdown may still severely affect adolescents due to the changes imposed on their normal daily routine [3,4,5]. These detrimental effects could even be amplified in adolescents who are at risk of developing psychopathology, because they could be particularly vulnerable for the negative effects of stress and the limited access to support or treatment [5,6,7,8].

Cross-sectional studies conducted at the start of the pandemic showed that adolescents reported experiencing anxiety, depression, and distress [9,10,11]. These symptoms were reported by parents as well, who also noted behavioral problems in their children and adolescents [12]. However, these studies either assessed the adolescents’ symptoms using unvalidated questionnaires or did not take into account the adolescents’ pre-pandemic levels of emotional and behavioral problems.

Studies that used mental health data collected before the onset of the pandemic have yielded conflicting results. For example, studies that combined data from adult cohorts showed that the presence of pre-existing mental symptoms was associated with an increase in both anxiety and depression during the pandemic [13]. In contrast, other cohort studies found that the increase in psychiatric symptoms was limited to adults without pre-pandemic symptoms, while adults with pre-existing symptoms did not report an increase in their symptoms during the pandemic [14]. In addition, among children and adolescents a psychiatric diagnosis prior to the pandemic was predictive of both improvement and a reduction in depressive, anxiety, and irritability symptoms experienced during the pandemic [15].

Determining the lasting effects of the pandemic among adolescents requires a longitudinal approach. However, the longitudinal studies reported to date primarily examined the effects of the first few months of the pandemic. For example, a comparison of two cross-sectional studies in China showed an increase in depression and anxiety from February 2020 to April 2020 [16]. In adults, a decrease in depression and anxiety was evident in the first few weeks of lockdown [17]; in children and adolescents, an immediate decrease in anxiety and depression was reported after the lockdown, although overall symptom levels remained high [18]. Finally, other studies found an overall decrease in psychiatric symptoms among children and adolescents [19, 20].

Importantly, the potential effects of the pandemic on severe psychiatric symptoms such as psychotic experiences and suicidality remain unclear. Although epidemics do not necessarily affect adolescent suicidality [8], admissions data from hospitals in the United States, Europe, and the Middle East showed an increase in self-harm, suicidal ideation, and suicide attempts during the COVID-19 pandemic [21, 22]. In addition, an online survey conducted during the pandemic revealed that 37% of adolescents reported having suicidal ideation [23].

In the Netherlands, the emerging pandemic led the Dutch government to implement far-reaching regulations, including the closing of schools, restaurants, and sports facilities, as well as a strict ban on public gatherings. To understand the extent to which the COVID-19 pandemic and the related societal changes affected the well-being of adolescents, their mental health status before the start of the pandemic must be taken into consideration. We therefore studied the prevalence of mental health problems in an at-risk cohort before the pandemic and during two nationwide lockdowns in the Netherlands. Specifically, we tested the hypothesis that during the lockdowns imposed during the COVID-19 pandemic, adolescents had higher scores related to anxiety, depression, stress, and oppositional defiance, particularly among adolescents who experienced emotional and behavioral symptoms prior to the pandemic. Furthermore, we hypothesized that suicidality and psychotic experiences increased during the lockdowns.

Methods

This study included two additional online questionnaires (C1 and C2, administered during the first and second nationwide lockdowns in April 2020 and January 2021, respectively) conducted within the iBerry (Investigating Behavioral and Emotional Risk in Rotterdam Youth) Study. The iBerry Study started in September 2015 and the study cohort has been described in detail previously [24]. In short, the iBerry Study is a population-based cohort study of adolescents designed to investigate the transition from subclinical symptoms to a psychiatric disorder. A cohort of high-risk adolescents was selected based on their self-reported emotional and behavioral symptoms experienced in the first year of high school (mean age 13.1 years). Each adolescent filled out the youth version of the Strengths and Difficulties Questionnaire (SDQ-Y), and adolescents with the highest 15% of scores and a random sample from the lowest 85% of scores were selected [24, 25]. The total cohort consists of 1022 adolescents, oversampled on their emotional and behavioral problems (2.5:1 ratio). At baseline (mean age 15.0 years), adolescents showed apparent emotional and behavioral symptoms; 30.7% of the high-risk adolescents reported emotional or behavioral problems above the borderline cut-off, compared to 8.1% of the low-risk adolescents [24].

A total of 36 adolescents (3.5%) were not invited to complete the initial online measurements because they objected to being approached for follow-up measurements, and an additional 54 adolescents (5.3%) could not be reached in order to obtain an email address and were therefore not included. The first questionnaire (C1) was sent on April 24, 2020, to the participating 932 adolescents during the first lockdown in the Netherlands, after Dutch high schools had been closed to in-class lessons (i.e., all students attended online classes only) for 6 weeks. The participants were able to complete the questionnaire during the 6-week period until several restrictions were lifted and high schools re-opened; 445 adolescents (48%) returned this questionnaire. The second questionnaire (C2) was sent to all adolescents who completed the first questionnaire; this second questionnaire was sent on January 8, 2021, during the second full lockdown in the Netherlands, after schools had been closed to in-class lessons for three weeks, and 333 adolescents (75%) completed the second questionnaire.

Measurements

Demographics

Several sociodemographic characteristics were obtained from the participating adolescents at baseline (i.e., prior to the pandemic), including age, gender, and education level. One of each adolescent’s parents provided data regarding ethnic background, household income, and urbanicity (based on the number of addresses per km2 surrounding their home address [24], with rural, suburban, and urban neighborhoods defined as < 1000, 1000–1500, and > 1500 addresses/km2, respectively).

Emotional and behavioral problems

To measure emotional and behavioral problems in the participating adolescents, we used several subscales of the Youth Self-Report (YSR) from the Achenbach System of Empirically Based Assessment (ASEBA) [26]. Specifically, the adolescents completed the following subscales: Depressive problems (12 items), Anxiety problems (9 items), Oppositional defiant problems (5 items), and Stress problems (14 items). We also used the YSR subscales Externalizing problems (32 items) and Internalizing problems (31 items) for supplementary analyses. At baseline participants were instructed to indicate whether each item was “Not true”, “Somewhat or sometimes true”, or “Very true or often true” when referring to problems experienced in the previous 6 months; for the C1 and C2 measurements, participants were instructed to respond based on problems experienced in the previous week. The YSR has good psychometric properties, including both reliability and validity [27]. All scales have norm scores, which were used to determine a borderline cut-off score (> 93rd percentile) and a clinical cut-off score (> 98th percentile) value based on a Dutch norm group [27]; scores above these cut-off values reflect symptoms that are considered to be of concern, indicating clinical relevance. The YSR total problem score at baseline was used to determine problem severity.

Psychotic experiences and suicidality

The sum score of two items (visual hallucinations and auditory hallucinations) in the YSR was used to measure whether the adolescent experienced psychotic symptoms. Similarly, the sum score of three items (self-harm, suicidal ideation, and suicide attempts) in the YSR was used to measure self-reported suicidality.

Statistical analysis

We examined the mean change in depressive problems, anxiety problems, stress problems, oppositional defiant problems, psychotic symptoms, and suicidality measured before the COVID-19 pandemic (baseline) and at C1 and C2. Multilevel random intercept regression models were estimated using a Restricted Maximum Likelihood estimator as implemented in the “lme4” package in R (v. 1.1.23) [28].

A set of three models with increasing complexity was fit to each outcome. The first model included only a random intercept to account for within-individual correlations and a time variable for changes between measurements. The second model included a random intercept in the mixed models in order to compare the changes in outcome scores from baseline to during the COVID-19 pandemic among various symptom severity groups. The interaction term between “time” and “group” indicated whether changes in symptoms differed between severity groups. In the third model, the following four demographic characteristics were added as covariates: age, gender, education level, ethnic background (Dutch or non-Dutch), and urbanicity (rural, suburban, or urban). We also obtained estimated marginal means in order to quantify changes in symptoms by severity group at baseline. Based on these estimated mean changes, we then calculated Cohen’s d effect sizes. Significance for these mean changes were determined using post hoc univariate linear regressions. For each model, variance explained by both the fixed and random effects was obtained using the conditional R2 metric [29].

Several supplementary analyses were also performed. We compared complete and incomplete case analyses and changed how baseline problem severity was operationalized in order to test the robustness of the interaction findings. We also used the baseline internalizing problems score for the anxiety, depressive, stress, suicidality, and psychosis outcomes. For oppositional defiant problems, we used the baseline externalizing problems score.

To determine whether missing data were missing at random, we used a univariate analysis to compare the participants who completed the C1 questionnaire with the participants who only completed the baseline questionnaire. For missing covariate data, we performed multiple imputation using the “mice” package in R (v. 3.11.0) [30].

Results

The C1 questionnaire was completed by 445 adolescents, 266 of whom (60%) were females. The majority (85%) of the participants had a Dutch ethnic background, and the mean age at C1 was 17.7 years (SD 0.67 years) (Table 1). At C1, 31 of the 445 participants (7%) had been positive for SARS-CoV-2 (the virus that causes COVID-19), and 138 participants (31%) reported having a relative and/or friend who was positive for SARS-CoV-2.

Table 1 Baseline sociodemographic characteristics of the 445 participants who subsequently completed the C1 questionnaire, stratified by the severity of baseline problems measured using the total emotional and behavioral problem score from the Youth Self-Report (YSR)
Full size table

Between baseline and C1, we observed an increase in depressive problems (B: 0.93, 95% CI [0.43, 1.42]). In contrast, anxiety problems (B: − 0.58, 95% CI [− 0.94, − 0.21]) and psychotic experiences (B: − 0.147, 95% CI [− 0.23, − 0.07]) decreased, whereas stress problems (B: 0.05, 95% CI [− 0.48, 0.59]), oppositional defiant problems (B: 0.30, 95% CI [− 0.18, 0.24]), and suicidality (B: − 0.05, 95% CI [− 0.13, 0.03]) were unchanged. Between C1 and C2, depressive problems (B: 2.20, 95% CI [1.71, 2.70]) and stress problems (B: 0.96, 95% CI [0.43, 1.50]) increased, psychotic experiences (B: − 0.13, 95% CI [− 0.21, − 0.05]) decreased, and anxiety problems (B: − 0.03, 95% CI [− 0.40, 0.34]), oppositional defiant problems (B: − 0.13, 95% CI [− 0.34, 0.08]), and suicidality (B: 0.03, 95% CI [− 0.04, 0.11]) did not change.

Figure 1 visually illustrates the percentage of participants at baseline whose symptoms for anxiety, depressive, oppositional defiant, and stress problems either changed (increased or decreased) or were unchanged from normal, borderline, or clinical at C1, as well as the percentage of participants at C1 whose symptoms either changed or were unchanged at C2. Between baseline and C1, 8.5%, 19.3%, 5.6%, and 11.4% of adolescents had an increase in anxiety, depressive, oppositional defiant, and stress problems, respectively, while 8.5%, 9.9%, 3.8%, and 7.8% of adolescents had a decrease in these subscales, respectively. Between C1 and C2, 10.0%, 20.5%, 2.6%, and 14.7% of adolescents had an increase in anxiety, depressive, oppositional defiant, and stress problems, respectively, while 4.4%, 7.3%, 4.7%, and 5.6% of adolescents had a decrease in these subscales, respectively. All other adolescents were unchanged (i.e., neither increased nor decreased) between baseline and C1 and/or between C1 and C2.

Fig. 1
figure 1

Percentage of adolescents whose emotional and behavioral symptom severity increased, decreased, or was unchanged from normal, borderline, or clinical severity between baseline and C1 and between C1 and C2. Note that the majority of adolescents were in the normal range throughout the study; to improve readability, the percentages in the normal range were scaled down by a factor of 10

Full size image

At C1, 88.5%, 74.6%, 93.7%, 85.1% of the adolescents scored in the normal range for anxiety, depressive, oppositional defiant and stress problems, respectively. Similarly, 83.9%, 64.5%, 95.6%, and 76.0% of the adolescents at C2 scored in the normal range for anxiety, depressive, oppositional defiant and stress problems, respectively.

Figure 2 summarizes the estimated mean subscale scores from the final model for all outcomes at baseline, C1, and C2, stratified by symptom severity at baseline. Based on the significant interaction of time and group, the change in the outcome scales between baseline and during the COVID-19 pandemic (with the exception of suicidality) differed among the various symptom severity groups at baseline. Supplementary Table S1 summarizes the coefficient estimates for all variables and outcomes. Our analysis showed that adolescents who scored in the clinical range at baseline had the largest decrease in problem scores between baseline and C1 for anxiety problems (Cohen’s d = 0.22), depressive problems (d = 0.12), oppositional defiant problems (d = 0.12), stress problems (d = 0.19), psychotic symptoms (d = 0.15), and suicidality (d = 0.11). The scores for these adolescents increased slightly between C1 and C2, albeit with small effect sizes, with Cohen’s d values ranging from 0.01 to 0.11. Adolescents who scored in the borderline range at baseline had a similar pattern as those who scored in the clinical range at baseline, although all effect sizes were small, with Cohen’s d values ranging from 0.00 to 0.07. Finally, adolescents who scored in the normal range at baseline had an increase in scores between baseline and C1 and again between C1 and C2, but all effect sizes were negligible, with Cohen’s d values ranging from 0.00 to 0.04 for all outcomes. The absolute estimated scores and Cohen’s d effect sizes for each group and outcome are summarized in Supplementary Table S2. The variance accounted for by the models ranged from 38% for suicidality and psychotic experiences to 60% for anxiety problems (see Supplementary Table S1).

Fig. 2
figure 2

Summary of the estimated scores for anxiety problems, depressive problems, oppositional defiant problems, stress problems, psychotic experiences, and suicidality reported at baseline, C1, and C2, stratified by emotional and behavioral problem scores measured before the COVID-19 pandemic. *p < 0.05 and **p < 0.001 (post hoc univariate test)

Full size image

In supplementary analyses, we observed only minimal changes in our estimates when we discarded incomplete cases rather than imputing the missing data, and when using internalizing problems rather than using total problems. Using externalizing problems rather than total problems for the outcome regarding oppositional defiant problems, however, increased the interaction effects with time. At baseline, the individuals in the clinical severity group for externalizing problems scored higher compared to the individuals in the clinical severity group for total problems (6.17 versus 4.52, respectively); this translated to a larger relative decrease and thus a larger standardized effect (Cohen’s d = 0.50), explaining the higher estimates of the interaction coefficient. Finally, a comparison between responders and non-responders with respect to the missing data revealed differences in demographic characteristics (i.e., gender, age, ethnic background, education level and household income) and in symptom severity at baseline (Supplementary Table S3).

Discussion

In this longitudinal study of adolescents, we found a small overall increase in depressive problems and stress problems during the COVID-19 pandemic. In contrast, we found that anxiety problems, oppositional defiant problems, psychotic experiences, and suicidality were either unchanged or decreased during the pandemic relative to pre-pandemic levels. Most notably, when we took the participants’ pre-pandemic mental health into account, we found a decrease in psychiatric symptoms among the participating adolescents with more severe emotional and behavioral problems prior to the pandemic.

These findings support different conclusions than most previous studies regarding the effects of the COVID-19 pandemic on mental health, that found mental health problems in adolescents [9,10,11,12]. Importantly, the majority of participating adolescents in our study reported symptoms in the normal range at both time points during the pandemic. As most studies report on findings in the general population, the seemingly different results in our study might actually agree with the results from the previously published studies, as we did find a slight increase in symptoms among adolescents with the least emotional and behavioral problems before the pandemic. The decrease of symptoms we found in adolescents with severe pre-pandemic symptoms was not reported in previous studies, possibly because of the lower prevalence of these symptoms in the general population. This illustrates the importance of taking the pre-existing characteristics of the study population into account when examining the effects of COVID-19.

Due to our observation that symptoms decreased among adolescents with the most severe pre-pandemic symptoms, we can reject our hypothesis that these adolescents were more vulnerable to the effects of the COVID-19 pandemic on their mental health. One possibility to explain this finding is that the restrictive measures may have led to adjustments that improved these adolescents’ symptoms, for instance fewer social obligations or smaller differences in experiences compared to less-vulnerable adolescents. Similar results were reported in psychiatric case–control cohort studies involving adults and elderly with depression and anxiety, in which individuals with the highest mental health burden tended to have a slight decrease in symptoms during the pandemic [14]. Similarly, young adolescents with high pre-pandemic symptoms reported a decrease in both internalizing problems and externalizing problems during the pandemic [31].

Our results should be interpreted in the context of the situation in the Netherlands [32]. In particular, the closure of schools and sports facilities is considered to significantly affect adolescents. On the other hand, the direct effect of COVID-19 on our participants was likely to be limited given the relatively small number of SARS-CoV-2 infections in our study population. Nevertheless, our findings suggest that the COVID-19 pandemic did not increase the participants’ symptoms above the clinically relevant threshold, an indication of adolescents’ resilience to the adverse effects of the pandemic [33, 34].

An important limitation of our study is the possibility of selection bias, as only 36% of our cohort completed both additional measurements (C1 and C2), resulting in relatively small sample sizes in the borderline and clinical severity groups. It is possible that the adolescents who did not participate experienced the most problems during the COVID-19 pandemic. However, a strength of our longitudinal design is the availability of extensive pre-pandemic data, including sociodemographic and symptomology data, which allowed us to identify key characteristics on which non-responders differed from responders. Being able to model these characteristics allowed us to move towards more unbiased estimates under the assumptions of missing at random; however, generalization of our findings should be done with caution [35]. Furthermore, our measurements, which were performed eight months apart during the pandemic, provide an indication of the effects of the ongoing pandemic on mental health among adolescents.

Lessons learned and consequences for the future

At the time of writing, the COVID-19 pandemic is ongoing and its full impact on the mental health of adolescents is yet to be determined. As the pandemic develops and stress accumulates, future preventive measures such as lockdowns could have qualitatively different effects on psychological symptoms. While the current results cannot be used to predict these future outcomes, they are relevant as lessons for studies conducted both later in or after the pandemic. The first lesson is that longitudinal studies with pre-pandemic measurements are necessary to appropriately assess changes in mental health, especially among vulnerable subgroups. Second, it is important to interpret results in the context of clinical relevance. Focusing on effect sizes and placing those findings within clinical thresholds will be essential in showing how the pandemic has affected the current generation of adolescents [36]. Consequently, it is important to be aware of our findings that suggest that the effect of the pandemic on mental health might be smaller than expected. Nevertheless, even small effects could be of importance at the population level. Future studies should carefully select representative samples with pre-pandemic measurements to identify both those at risk of negative consequences of the pandemic as well as those that show resilience.

Importantly, we were able to study adolescents during a period marked by significant changes both developmentally and due to the COVID-19 pandemic. The majority of the participating adolescents reported symptoms that were within the normal range. Our findings underline the importance of considering pre-pandemic mental health, as a decrease in symptoms was apparent in adolescents with more severe pre-pandemic emotional and behavioral problems while adolescents without significant pre-pandemic problems showed a small increase in depressive and stress problems. As the COVID-19 pandemic continues and more studies investigate its long-term effects, attention should be paid to the pre-pandemic health of study participants to understand how lockdown measures affect adolescent mental health.