Introduction

The COVID-19 pandemic had a dramatic and unprecedented effect on the US education system, with the majority of schools closing down in March of 2020 (Donohue & Miller, 2020). While these closures were executed based upon public health guidance to protect students and staff, the consequences of remote instruction and lack of access to student support services for over two years were largely unknown (Viner et al., 2020). In the USA, marginalized communities and those living in poverty were most impacted by the disparities in access to digital technologies needed for remote instruction, in addition to a lack of access to health care services, subsidized or free meals, and adequate childcare when schools closed (Armitage & Nellums, 2020). In addition, many students faced increased stressors and hardships as pre-existing circumstances such as financial, housing, and food insecurities were exacerbated (Minkos & Gelbar, 2021; Van Lancker & Parolin, 2020). Thus, after weighing the risks of the virus with the adverse effects of remaining closed, many school districts made the difficult decision to return to in-person learning with additional preventive measures such as social distancing and increased cleaning.

The disruptions associated with the COVID-19 pandemic are likely to have significant, long-term impacts on a variety of youth outcomes (Azevedo et al., 2021; Engzell et al., 2021). Early research indicates decreased rates of focus, attention and sleep, and increased clinginess, fear, and irritability among youth (Hamilton & Gross, 2021; Larsen et al., 2021; Singh et al., 2020; Viner et al., 2020). The impact of the COVID-19 pandemic and related school closures also disrupted school-related supports and services that are fundamental to children's development and well-being. Dudovitz and colleagues (2022) conducted a study with 1,504 US parents to determine the impact of the pandemic on the social-emotional well-being and educational needs of their school-aged children during the 2021–2022 academic year. Results indicated that 26.1% of children had challenges with hyperactivity/inattention, 32.6% had peer relationship problems, and 40% had deficits in prosocial behaviors. The majority of parents (83.5%) reported a school-related need, with 57% reporting mental health challenges and 77% reporting learning supports and enrichment needs. Parents reported their child’s highest priority needs to be for tutoring, socialization, increased instructional time, managing stress, and physical activity (Dudovitz et al., 2022). As indicated by these findings, student well-being and interpersonal relationship skills were significantly impacted by the pandemic as well as students’ ability to cope with stressors. Although some research suggests that bullying behaviors decreased during the pandemic (Bacher-Hicks et al., 2022), additional stressors like food insecurity and financial stressors on family well-being place children at greater risk for bullying involvement (Malecki et al., 2020; Steimle et al., 2021; Woolweaver et al., 2022).

To address these deficits and mitigate current and future harm, schools must implement programming that supports students' social-emotional learning (SEL). Hamilton and Gross (2021) found that in the Fall of 2020, 66% of schools prioritized SEL in their reopening plans to address these gaps. SEL programs that focus on building relationships, emotional well-being and resilience have been linked to a variety of beneficial outcomes for youth, including positive behavioral and academic outcomes (Taylor et al., 2017). Research suggests one of the primary strategies to prevent and reverse the negative effects of this pandemic and support student success is by fostering resilience (Jiao et al., 2020). Students need additional instructional support in developing and practicing these skills through structured school-based SEL programming.

The Sources of Strength Elementary Program

Sources of Strength is an evidence-based SEL suicide prevention program originally developed for middle and high school students (Wyman et al., 2010). The program's overarching objective is to promote and develop protective factors and integrate resilience into the school community, culture, and climate. Figure 1 shows the eight focal protective factors for youth and adolescents that are central to the program: family support, positive friends, mentors, healthy activities, spirituality, generosity, physical health, and mental health. In the middle and high school programs, student peer leaders are trained to recognize, build, and harness resilience through the eight protective factors while working with adult advisors to promote help-seeking behaviors among their peers. In a randomized clinical trial with 18 high schools, schools with Sources of Strength reported greater increases in student-adult connectedness, school engagement, and help-seeking acceptability compared to control schools, all factors that were associated with reductions in suicidal ideation (Wyman et al., 2010). In 2011, the program was placed on the Substance Abuse and Mental Health Services National Registry for Evidence-based Programs and Practices (NREPP; Suicide Prevention Resource Center, n.d). Although most suicide prevention programs are designed for adolescents, research indicates that preadolescents are also experiencing suicidal thoughts and behaviors at alarming rates, calling for resilience and suicide prevention among elementary school students (Gallo et al., 2021; Miller, 2019).

Fig. 1
figure 1

Sources of strength wheel

Full size image

In response to the need for universal upstream suicide prevention and SEL programming in elementary schools, Sources of Strength Elementary was developed in 2020 and implemented for the first time during the 2020–21 academic school year (https://sourcesofstrength.org/elementary/). The elementary school curriculum takes a strengths-based approach by promoting resilience through social and emotional learning principles (self-awareness, social awareness, self-regulation, responsible decision-making and relationship skills), and the eight protective factors are shown in Fig. 1. The Sources of Strength Elementary curriculum was iteratively developed with feedback from teachers, counselors, and SEL specialists. Sources of Strength Elementary utilizes an evidence-based transformational coaching model (Aguilar, 2013), which promotes adult SEL by empowering staff to understand and disseminate material to other school staff through professional development and to embed the curriculum’s concepts (i.e., building resilience and social-emotional competencies, etc.) into school culture.

Theoretical Framework

The Sources of Strength Elementary curriculum is informed by social learning theory, which proposes that social behaviors and skills can be acquired by observing and imitating others (Bandura & Walters, 1977). The curriculum utilizes SEL principles to build resiliency at the individual level. The learn, apply, invite model allows implementers (i.e., teachers and other school staff) to learn the concepts, apply them in their own lives, and then invite others to join them. This format allows curriculum coaches and instructors to benefit directly from the Sources training and indirectly through interactions with their students while implementing the curriculum. This curriculum also takes an evidence-informed, public health upstream prevention approach by promoting associated social-ecological protective factors at individual, relational, classroom, and school community levels to target key outcomes (academic, social-emotional, and behavioral outcomes; David-Ferdon et al., 2016). However, the most significant benefits of this model are expected among students because they are at a critical point in development, where SEL skills can grow significantly and have lasting effects. In addition to improving students’ and teachers’ skills, SEL programming can improve the classroom climate.

Despite the many potential positive outcomes associated with strong social-emotional skills and high-quality SEL programming, implementation fidelity varies widely, as do subsequent outcomes (Durlak, 2016). As a result, many universal SEL programs in the USA report modest or no effects, while other studies find strong effects for certain programs or populations (Taylor et al., 2017). According to the diffusion of innovation theory, an intervention's effectiveness depends on the intervention's diffusion and exposure to as many students and staff in a school building (Rogers, 1995). Thus, to evaluate the effectiveness of the Sources of Strength Elementary curriculum, it is critical to consider the extent to which its effectiveness varies as a function of youths’ exposure to the program messaging.

The Current Study

The current study evaluates the effectiveness of the Sources of Strength Elementary program on the following student social-emotional outcomes: positive classroom climate, emotional problems, school belonging, help-seeking attitudes, bullying perpetration, peer victimization, students' and teachers’ likelihood to intervene in bullying situations, student well-being, and student resilience. Further, we examine the associations between greater student exposure of the Sources program and each of these outcomes.

We hypothesize that students who participated in the program and those with a higher program exposure would have lower levels of emotional problems, bullying perpetration, and peer victimization at Time 2 when compared to Time 1. Additionally, we hypothesize that study participants and students exposed to higher levels of the program would be associated with higher levels of positive classroom climate, school belonging, help-seeking attitudes, student intervention, teacher intervention, student well-being, and student resilience at Time 2 when compared to Time 1.

Methods

Participants and Procedures

The current study used data collected from 3rd to 5th grade students (N = 1022) in 62 classrooms across 11 elementary schools in the Great Plains region of the USA. Baseline data were collected during the Fall of 2020, in collaboration with the public school system and the Sources of Strength suicide prevention program (LoMurray, 2005; Sources of Strength, 2021). Primary data were collected by classroom teachers, and secondary data analysis of de-identified data was approved by district administrators and the partnering university IRB.

School District Context

Due to the variability in instruction as a response to the COVID-19 pandemic, it is important to understand the context of the schools in this study. According to publicly available data from the district website and contextual information provided by the district, remote instruction began in March of 2020. The district implemented safety, sanitation, and social distancing measures to combat the pandemic, including regular disinfection of surfaces, the requirement of masks and a mandatory quarantine period for those exposed to COVID-19. In July 2020, parents were given the option to have their child return to school or continue remote learning. In August 2020, only 610 elementary students (8%) in the district were learning remotely. At the time of survey administration (T1) in October 2020, the district alerted parents that they were heading into the severe risk category and would closely monitor the situation in the school and community. Remote learning began again in November 2020, with a plan to return to in-person in January 2021. Between October and November 2020, the student and staff COVID-19 infection rate ranged from 1.4 to 3.3%. In January 2021, when school started, there were options for in-person or remote learning. The district has remained in-person since, including during survey administration in March 2021 (T2), when the student and staff infection rate was 0.2%.

Implementation of Sources of Strength Curriculum

Four units of the Sources curriculum were implemented, which consisted of a total of 14 lessons (See Table 1 for descriptions of each unit). Lessons are designed to be 30 min in length and are typically facilitated one or two times per week. The driving principle of Sources is incorporating student voice and active engagement through interactive teaching practices. Each of the 4 units has seven specific components: (1) instructor participation, (2) active learning, (3) talking circles, (4) guided reflection, (5) celebrations of growth, (6) everyday applications, and (7) social-emotional focus. Instructors consist of teachers who work with coaches (i.e., school counselors). Coaches and instructors met weekly in small groups to discuss their experiences and progress with the curriculum.

Table 1 Sources of strength lessons
Full size table

In response to the unexpected closures and challenges related to COVID-19, the Sources team has responded by modifying content to fit an online mode of delivery, providing additional educational content to assist with this transition, and reaching out to all school partners to show them how to navigate these tools. By providing free access to a variety of resources, including games, lessons, wellness plans, etc., the Sources team assisted students, families, and school staff in their growth and development, even in times of uncertainty, and demonstrated their capacity to provide high-quality and flexible programming.

Measures

Student constructs were assessed through a combination of existing and newly developed measures. Questions were developed by a team of researchers, former elementary teachers, and Sources personnel who informed the selection of developmentally appropriate items for students in grades 3–5. To inform future program development, the measures created for this study were designed to maximize fit with the Sources target outcomes. Before completing the survey, students answered an example question about ice cream preferences to confirm student understanding of the functionality of the survey platform and Likert-type scales. Questions were displayed as multiple choice or Likert-type scales, with minimal questions on each page to limit the visual and reading burden. Students read and completed the survey independently with support from teachers as needed.

Positive Classroom Climate

Positive classroom climate was measured with eight items adapted from the School Sense of Belonging Scale (Goodenow, 1993). Students were asked how much they agree with statements such as “Students in my class get along with each other,” “My classroom is a good place to be,” and “I trust my teachers in my classroom.” Response options were 1 (Almost never), 2 (Sometimes), and 3 (Most of the time). The construct validity and reliability of the scale were tested in a previous study using Confirmatory Factor Analysis (CFA), which showed an adequate model fit to this data (Drescher et al., 2022). The Cronbach’s alpha coefficients for this study were α = .70 and .75 for T1 and T2, respectively.

Emotional Problems

Twelve items developed by the Sources of Strength team and school district administrators were used to measure emotional problems. Students were presented with the following prompt: “The following questions ask about emotions. For example, you feel emotions when you feel happy, sad, or angry. How often do you feel this way at school?” Example items included: “I feel sad,” “I feel worried,” and “I lose my temper.” Response options were 1 (Almost never), 2 (Sometimes), and 3 (Most of the time). The factor structure for the scale showed adequate fit to the data in a CFA conducted by a previous study (Drescher et al., 2022). The Cronbach’s alpha coefficients for this study were α = .81 and .82 for T1 and T2, respectively.

School Belonging

Twelve items developed for this study were used to measure students' school belonging. Students were asked to indicate how true were statements such as “I get along with people at school,” “I feel like my classmates care about me,” and “I like school.” Response options were 1 (Not at all true), 2 (A little true), and 3 (Very true). The Cronbach’s alpha coefficients for this study were α = .79 and .83 for T1 and T2, respectively.

Help-seeking Attitudes

Seven items adapted from the Help-seeking From Adults at School scale (Wyman et al., 2010) were used to measure students’ attitudes toward help-seeking. Students were asked how true were the following, example items included: “I would ask for help if I was hurt or upset,” “I would ask my teacher for help if I needed it,” and “I would get help for a student being picked on.” Response options were 1 (Not Really True), 2 (A little true), and 3 (Definitely true). The Cronbach’s alpha coefficients for this study were α = .75 and .81 for T1 and T2, respectively.

Bullying Perpetration

Four items adapted from the University of Illinois Bully Scale (Espelage & Holt, 2001) were used to measure bullying perpetration. Students were asked to indicate how often they had done the following in the past month, (1) “I pushed or tripped a student,” (2) “I said mean things to other students,” (3) “I spread rumors about other students,” and (4) “I used the internet to tell lies or make fun of other students (for example: email, instant messaging, text messaging, or websites).” Response options were 1 (Never), 2 (Once or twice), 3 (Several times), and 4 (A lot). The construct validity of this scale has been supported via exploratory and confirmatory factor analysis (Espelage & Holt, 2001; Espelage et al., 2003). The Cronbach’s alpha coefficients for this study were α = .68 and .69 for T1 and T2, respectively.

Peer Victimization

The University of Illinois Victimization Scale (Espelage & Holt, 2001) was used to measure peer victimization. Students were asked to indicate how often in the past month had another student or group of students, (1) “pushed or tripped me,” (2) “said mean things to me,” (3) “spread rumors about me,” and (4) “told lies or made fun of me using the internet/cell phone (for example: email, instant messaging, text messaging, or websites).” Response options were 1 (Never), 2 (Once or twice), 3 (Several times), and 4 (A lot). The scale has shown high construct validity and reliability in previous studies (Espelage & Holt, 2001; Espelage et al., 2003). The Cronbach’s alpha coefficients for this study were α = .80 and .78 for T1 and T2, respectively.

Student and Teacher Intervention

Five items adapted from the Teacher and Staff School Environment Survey (Espelage et al., 2014) were used to measure students' and teachers’ likelihood to intervene in bullying situations. Students were asked to indicate whether most students or teachers in their class would help out if the following happened during the past month. Example items include (1) “A student teases another student,” (2) “A student spread rumors about another student behind their back,” and (3)“A student or group of students pushes or tries to pick a fight with another student.” Response options were 1 (Almost never), 2 (Sometimes), and 3 (Most of the time). The Cronbach’s alpha coefficients for the student intervention scale were α = .91 and .88 for T1 and T2, respectively. The alphas for the teacher intervention scale were α = .95 and .91 for T1 and T2, respectively.

Student Well-Being

Nine items adapted from the General Well-Being Scale (Wyman et al., 2010) were used to measure student well-being. Students were asked how true were the following, example items included: “I am a healthy person,” “I have hobbies and activities that I enjoy,” and “I have positive friends in my life.” Response options were 1 (Not Really True), 2 (A little true), and 3 (Definitely true). The scale has demonstrated high reliability and adequate fit to the data in previous studies (Drescher et al., 2022; Valido et al., 2021; Wyman et al., 2010). The Cronbach’s alpha coefficients for this study were α = .76 and .83 for T1 and T2, respectively.

Student Resilience

Ten items developed for this study were used to measure student resilience. Students were asked how true were the following, example items included: “When hard things happen I feel like I know how to get through it,” “It is ok to make mistakes,” and “Things can get better.” Response options were 1 (Not Really True), 2 (A little true), and 3 (Definitely true). The Cronbach’s alpha coefficients for this study were α = .82 and .83 for T1 and T2, respectively.

Sources of Strength Program Exposure

Four items developed for this study were used to measure student’s exposure to the Sources of Strength program at T2. Students were presented with the statement “Outside of the Sources of Strength lessons my friends/classmates…,” (1) “Talked about strengths,” (2) “Talked to me about using my strengths,” (3) “Used what we learned in the lessons to deal with big emotions,” and (4) “Used the regulation railroad to help with their feelings.” Response options were 1 (No) and 2 (Yes). The scale was computed by averaging across the four items. The Cronbach’s alpha coefficient for this study was α = .79 for T2.

Qualitative Feedback About the Sources Program

To contextualize the quantitative findings of this study, we collected qualitative data from teachers in a separate online survey regarding their experiences with the implementation of the Sources of Strength program in their classrooms. Classroom teachers were asked to provide their comments in a series of open-ended questions embedded in the survey. Example questions included: “Is there anything else you would like to say about the Sources of Strength program or how it impacted your students/classroom?” and “Please explain your experience with this implementation method.”

Analysis Plan

To determine whether the Sources of Strength Elementary program had an overall effect on student outcomes, we used a series of multilevel models where the student self-reported program exposure measured at Time 2 was regressed on each outcome. Each multilevel model was run with the following covariates: other sources of change in the outcomes between Time 1 and Time 2, race/ethnicity (Black, Hispanic, Other race compared to White), age (years), and gender (female compared to male).

We first calculated intra-class correlations (ICCs) for the school and classroom levels to determine whether multilevel modeling was needed to account for the nested nature of the data. Researchers recommend against ignoring the multi-level nature of the data when data are nested, because even small ICCs (> .01) at the group level can produce biased estimates (Clark, 2008; Musca et al., 2011; Nezlek, 2008). Although there are no clear guidelines for when the number of level 2 units is large enough to conduct a multi-level analysis, given the sample of 11 schools and 62 classrooms, we considered it important to account for this level of analysis if the ICCs indicated a variation at the school and classroom level of at least ICCs > .03, which is relatively small but non-ignorable for the social sciences. However, due to the limited sample of schools, we chose to only include random intercepts to account for school-level variability and account for the potential bias introduced by ignoring the nesting structure, but no inferences were made at the school and classroom levels.

All descriptive analyses were conducted using the R open-source statistical software and multilevel analyses were conducted using the lme4 R package (Bates et al., 2015). Missing data ranged from 1.1 to 11.2% at Time 1 and from 5.2 to 16.2% at Time 2. We conducted a logistic regression to evaluate whether students with missing data in the outcomes of interest differed according to race, gender, age, exposure to sources of strength, or the time of survey. The analysis suggested that female students and older students had a lower percentage of missing data, as were those that answered the survey at Time 2. However, students with missing data did not significantly differ according to race/ethnicity or exposure to the Sources program.

To account for the potential bias introduced by missing data, we used multiple imputation with the R package MICE (Multivariate Imputation by Chain Equations; Van Buuren & Groothuis-Oudshoorn, 2011). All analyses were conducted with 100 datasets with imputed missing data, which included the school and classroom level clusters as auxiliary variables to account for the associations within clusters. The final parameters were an average of the parameter estimates across the 100 datasets. Further, we conducted a sensitivity analysis comparing the parameters resulting from the multiple imputation analysis and those generated by running the models without handling the missing data. Patterns of significant findings remained the same, suggesting the multiple imputation adequately captured the original distribution (Table 2).

Table 2 Descriptive statistics for all scales
Full size table

Though qualitative comments from teachers were not the focus of the study and are only presented in supplemental form to provide some context about teachers' perspectives regarding the program implementation, these comments were chosen and analyzed using a modified thematic analysis framework (Braun & Clark, 2006). The research team first familiarized themselves with the qualitative data. Then, individually, each researcher selected the most salient quotes related to teachers’ experience with Sources of Strength and its implementation. Researchers met to discuss chosen quotes and, as a group, decided which quotes to present. This selection of quotes from teachers can be found in Supplemental Table S1.

Results

Descriptive statistics and sample characteristics are presented in Table 2.

Intra-class Correlations (ICC)

Analysis of intra-class correlations revealed that a small to moderate amount of the variance in student outcomes could be attributed to the school and classroom levels (Table 3). The ICCs varied from .01 to .06 at the school level and from .06 to .12 at the classroom level. Given that substantial variation was due to the school and classroom contexts, we decided to include random intercepts for the school and classroom in the multilevel analyses.

Table 3 Summary of unconditional intra-class correlations (ICC) at the school and classroom levels
Full size table

Sources of Strength Program Exposure

Greater student exposure to the Sources program was significantly associated with several positive outcomes (Table 4). Higher self-reported exposure was associated with higher reports of positive classroom climate (b = .21, SE = .03, p < .001), lower reports of emotional problems (b = − .09, SE = .04, p < .05), higher levels of school belonging (b = .22, SE = .03, p < .001), higher levels of help-seeking attitudes (b = .38, SE = .04, p < .001), higher levels of well-being (b = .27, SE = .03, p < .001), and increased resiliency (b = .30, SE = .03, p < .001).

Table 4 Multilevel model results
Full size table

Changes between Time 1 (T1) and Time 2 (T2)

Despite improvements among students with higher program exposure, the multilevel models also showed significant declines in the target outcomes at T2 when compared to T1 (Table 4). At T2, students reported lower positive classroom climate (b = − .38, SE = .05, p < .001), higher reports of emotional problems (b = .20, SE = .06, p < .001), lower school belonging (b = − .37, SE = .05, p < .001), lower help-seeking attitudes (b = − .62, SE = .06, p < .001), lower well-being (b = − .44, SE = .05, p < .001), and lower resiliency (b = − .47, SE = .05, p < .001).

Race/Ethnicity

Overall, students with minority race/ethnicities reported worse levels of each outcome when compared to their White peers (Table 4). Students who identify as Hispanic students reported lower positive classroom climate (b = − .19, SE = .03, p < .001), higher emotional problems (b = .18, SE = .04, p < .001), lower school belonging (b = − .16, SE = .03, p < .001), lower help-seeking attitudes (b = − .19, SE = .04, p < .001), higher bullying perpetration (b = .06, SE = .03, p < .05), higher peer victimization (b = .16, SE = .05, p < .01), lower well-being (b = − .13, SE = .03, p < .001), and lower resiliency (b = − .17, SE = .04, p < .001). Black students reported lower positive classroom climate (b = − .07, SE = .03, p < .05), higher emotional problems (b = .12, SE = .04, p < .001), lower help-seeking attitudes (b = − .12, SE = .04, p < .001), higher bullying perpetration (b = .08, SE = .03, p < .01), and lower resiliency (b = − .07, SE = .03, p < .05). Similarly, students who identified as other race (Asian, Native American, Pacific/Islander, and other) reported lower positive classroom climate (b = − .07, SE = .02, p < .001), higher emotional problems (b = .11, SE = .03, p < .001), lower school belonging (b = − .09, SE = .02, p < .001), lower help-seeking attitudes (b = − .07, SE = .03, p < .05), higher peer victimization (b = .10, SE = .04, p < .01), lower perceptions of teacher intervention (b = − .14, SE = .04, p < .001), lower well-being (b = − .07, SE = .02, p < .001) and lower resiliency (b = − .08, SE = .02, p < .001).

Age

Overall, older students reported significantly lower levels of several outcomes when compared to younger students. Older students were associated with lower reports of school belonging (b = − .03, SE = .01, p < .05), lower help-seeking attitudes (b = − .03, SE = .01, p < .01), and lower perceptions of student intervention (b = − .06, SE = .02, p < .05).

Gender

Lastly, when compared to male identified students, female identified students were associated with higher reports of positive classroom climate (b = .04, SE = .02, p < .05), higher school belonging (b = .06, SE = .01, p < .001), higher help-seeking attitudes (b = .11, SE = .02, p < .001), lower bullying perpetration (b = − .09, SE = .01, p < .001), lower peer victimization (b = − .05, SE = .03, p < .05), higher perceptions of student intervention (b = .16, SE = .03, p < .001), and higher perceptions of teacher intervention (b = .16, SE = .03, p < .001).

Qualitative Comments

Qualitative comments suggest that teachers had a positive perception of the Sources program in their classrooms. Some teachers indicated using the concepts and ideas from the program in their everyday conversations and activities with students. Others highlighted how the Sources program filled the need for social-emotional learning programs and support for students at the elementary level. Although most teachers perceived the program as acceptable and effective in engaging students, others did not perceive noticeable changes in their classrooms. In general, teachers had a positive perception of the implementation and described the program as a helpful resource for their students.

Discussion

The 2020–2021 academic year context presented numerous challenges that shaped student learning and behavior. Although our findings showed significant declines in student outcomes when comparing pre- and post-measurements, we found that greater student exposure to the Sources program was associated with increased positive perceptions of classroom climate, lower reports of emotional problems, higher perceptions of school belonging, improved attitudes toward help-seeking, higher well-being, and gains in resiliency at post-measurement (controlling for baseline). These findings suggest that students appeared to benefit when they are familiar with and use strategies from the program. Given we did not measure implementation fidelity in each classroom, the lack of positive effects when comparing pre- and post-student outcomes could be a result of fidelity variability. The importance of implementation fidelity for the effectiveness of educational programs has been highlighted in the previous literature (Durlak, 2016; Elias et al., 2003; Fisher et al., 2014; McKenna et al., 2014). However, the negative findings observed in the pilot study could also be due to contextual effects occurring during the implementation and the challenges of implementing the program during a pandemic, a time of heightened stress for teachers, students, and families.

It is important to recognize that students' participation in this program is not occurring in a vacuum and a broad range of external experiences are also affecting student outcomes. First, data were collected at two time points, each at a unique stage in the pandemic. During Time 1 in Fall 2020, many parents may have still been home with their children, potentially facilitating a climate of well-being, help-seeking and other forms of emotional support at home, whereas greater levels of disruption were occurring in family life as schools began to reopen in 2021. Additionally, decreases in school belonging and classroom climate could be related to decreased connectedness as a result of the increased variation in school routines between in-person, online, and hybrid learning styles. Many students had only returned to in-person instruction a couple months before the survey administration which may not have left sufficient time for them to develop meaningful relationships with their teachers and classmates.

The added stressors associated with the pandemic on students and teachers during this period emphasize the need for SEL programs that can alleviate these negative outcomes. The positive associations of program exposure with classroom climate, emotional problems, school belonging, help-seeking, well-being, and student resiliency indicate promise for the Sources of Strengths Elementary curriculum. Findings suggest that strength-based SEL programs, such as Sources of Strength Elementary, can positively impact student outcomes. However, ensuring implementation fidelity is crucial for program effectiveness. These results were strengthened by positive comments from teachers who perceived the program as helpful and effective in their classrooms despite the challenges of implementing it in in-person, online, and hybrid environments.

Limitations

This Sources of Strength Elementary pilot evaluation provides practical and valuable data regarding the implementation and efficacy of an SEL curriculum within elementary schools. Although this study showed promise for Sources’ impact on improving students’ emotional health during uncertain times like the COVID-19 pandemic, this study was limited in several respects. The most significant limitation was simply conducting this evaluation during COVID-19, when students, teachers, and their families may have already been struggling academically and emotionally, and these challenges were likely exacerbated during the pandemic. This is particularly concerning because we did not have a control condition through a randomized clinical trial. The lack of a control group precludes any indication of causality; therefore, we recommend caution when interpreting these findings. However, the ethical decision to not withhold intervention through a controlled trial during a pandemic outweighs the need for causal claims. Furthermore, this was a small pilot study that consisted of 11 schools with a majority of White students and teachers in one region of the USA; thus, generalizability is limited to this demographic. Therefore, the data are not representative of the diverse student and teacher populations in other areas of the USA. Further, given the small number of schools, this study was underpowered at the school level to uncover significant associations; however, we choose to account for the nesting structure given the relatively high ICCs uncovered at the school level which could lead to bias estimates. We attempted to account for that variation by conducting sensitivity analysis, using multiple imputation, and multilevel analysis but we caution against making any inferences at the school or classroom level. In addition, all measures were self-reported by third through fifth graders, and data were collected by their classroom teachers. Though the survey was developmentally appropriate, there may still be bias due to the self-reported nature of the responses and the collection of data by teachers.

Implications for Social-Emotional Programs

The COVID-19 pandemic immensely impacted students’ mental health across the USA. Rates of stress, anxiety, and depression in children skyrocketed as they transferred to online-based learning in early 2020 (Reinert et al., 2021). These rates have only increased as social isolation, fear of sickness, and economic turmoil continued into 2021 (Reinert et al., 2021). This increase in mental health issues has spotlighted the emotional and social needs of children in academic and home settings. It is well established that SEL programs increase protective factors against negative mental health outcomes (Durlak et al., 2011; Jones et al., 2015; Taylor et al., 2017). Children introduced to primary prevention programs early on that focus on enhancing resilience and self-regulation, both standards of SEL curriculum, apply these skills at a higher rate to problem situations or negative life experiences later on (Jones et al., 2019). Situations like the unpredictable COVID-19 pandemic have shown the necessity of such educational programs (Espelage et al., 2016).

For this reason, as schools look to increase their student’s social-emotional competencies, there may be an increased diffusion of SEL curricula across the USA. Incorporation of these curricula into the academic standards for preschool through primary school would be highly beneficial to children’s long-term well-being. They must emphasize the five interconnected sets of core cognitive, affective, and behavioral competencies: self-awareness, self-management, social awareness, relationship skills, and responsible decision-making (Atwell & Bridgeland, 2019). Given the value of social-emotional competencies and their relationship to promoting universal prevention strategies, program faithfulness to the goal of student well-being and consistency across time is essential.

Further research into transferring SEL programs faithfully into online learning platforms is needed as schools continue to further incorporate technology in the classroom. The Sources of Strength Elementary program responded to school closures in 2020 by modifying content to fit an online mode of delivery and providing additional educational content to assist with this transition. Other SEL programs can reference successful curricula that engage children through technology for schools to leverage in the future. The diffusion of social-emotional learning through these platforms has the opportunity to reach large numbers of students who may not otherwise come into contact with such programs.

Conclusion

The evaluation of an intervention’s effectiveness hinges on a wide variety of factors including context and implementation fidelity or exposure of the intervention. The Sources of Strength Elementary school curriculum was implemented in 3rd–5th grade classrooms during the COVID-19 pandemic, and as a result, students experienced variability in their exposure to the curriculum content. When examining the intervention effect during the pandemic, students appeared to have worse outcomes. However, when the level of student exposure was considered, the results suggest that greater student exposure to the Sources of Strength Elementary curriculum was associated with improved outcomes. Thus, greater exposure to Sources of Strength may be considered a promising SEL intervention for improving individual, classroom, and school-level protective factors in elementary school students. However, additional research is warranted to determine the efficacy of the program.