Social and emotional skills refer to the ability to coordinate cognition, affect, and behavior. These skills allow individuals to accomplish specific tasks in diverse social contexts and achieve positive developmental outcomes (Mahoney et al., 2021). Research showed that the improvement of social and emotional skills is critical for students’ cognitive development and well-being (Durlak et al., 2011; Taylor et al., 2017), as well as for facing the increasing demands and challenges of today’s society (Elias et al., 1997; Cristóvão et al., 2017). Research also showed that social and emotional skills are malleable and teachable (Jones & Kahn, 2017). They can be successfully developed and promoted through high-quality social and emotional learning (SEL) programs (Durlak et al., 2011; Taylor et al., 2017), particularly in educational contexts.

According to the Collaborative for Academic, Social, and Emotional Learning (CASEL, 2015), SEL refers to:

The process through which children and adults acquire and effectively apply the knowledge, attitudes, and skills necessary to understand and manage emotions, set and achieve positive goals, feel and show empathy for others, establish and maintain positive relationships, and make responsible decisions. (p. 5)

CASEL is a leading organisation in the US responsible for planning, implementing, and evaluating SEL programs. The CASEL framework was developed by researchers in the field of SEL in response to the growing recognition of SEL importance in academic success and overall well-being. This framework was aimed to provide a comprehensive structure for promoting SEL skills and its development relied on evidence-based practices and input from experts in education, psychology, and related fields (Durlak et al., 2011). CASEL proposed five interrelated SEL core competencies: (a) self-awareness, which consists in the ability to recognize one’s own emotions and accurately assess one’s own strengths and weaknesses; (b) self-management, which is the ability to regulate thoughts, emotions, and behaviors; (c) social awareness, which includes awareness of the culture, beliefs, and feelings of other people and the world; (d) relationship skills, which is the ability to communicate effectively, work well with peers, and build meaningful relationships; and (e) responsible decision-making, which consists in the ability to make plans for the future, and follow moral and ethical standards (Oberle et al., 2016). These five SEL competencies are conceptually aggregated into three broader categories: (a) intrapersonal skills, which include self-awareness and self-management; (b) interpersonal skills, which include social awareness and relationship skills; and (c) responsible decision-making (CASEL, 2013). Intrapersonal skills involve knowing one’s own feelings and values, cultural identity, sense of purpose, perseverance, and goal setting. Interpersonal skills encompass empathy, compassion, collaboration, and leadership. Responsible decision-making includes making ethical and principled choices in personal and social situations and it is considered both an intrapersonal and interpersonal competency (Cristóvão et al., 2017; Elliott et al., 2021; Mahoney et al., 2021).

Several studies showed that the five SEL competencies are associated with positive developmental trajectories, by contributing to the achievement of positive academic, social, and mental health outcomes (Durlak et al., 2011; Taylor et al., 2017). For instance, cultivating self-awareness allows individuals to recognise and understand their emotions, thoughts, and values, which promotes academic performance, interpersonal relationships, and psychological well-being (CASEL, 2013). Similarly, improving self-management allows individuals to regulate their emotions, set and achieve goals, and maintain concentration and resilience. This boosts academic performance and reduces behavioural problems (Durlak et al., 2011). In addition, promoting social awareness cultivates empathy, perspective-taking, and appreciation for diversity, fostering inclusive and supportive school environments conducive to academic engagement and positive social interactions (Durlak et al., 2011). The improvement of relational skills facilitates effective communication, collaboration, and conflict resolution, promoting healthy peer relationships and teacher-student interactions that contribute to academic success and emotional well-being (Jones & Bouffard, 2012). Furthermore, promoting responsible decision-making allows individuals to make informed choices, consider ethical implications, engage in critical thinking, control impulses, and evaluate the consequences. As a result, individuals show adaptive behaviours, enhanced problem-solving skills. and reduced risk behaviours, which ultimately support positive academic, social, and mental health outcomes (Durlak et al., 2011).

These five SEL competencies can be developed through SEL programs. Various studies showed that effective SEL programs may promote academic achievement, school adjustment and involvement, school well-being, and positive social relationships, thereby contributing to peer acceptance, pro-social behaviours, effective conflict resolution, and better-quality relationships (Blair & Raver, 2015; Corcoran et al., 2018; Durlak et al., 2011; Gresham et al., 2020; Jones et al., 2017; Jones & Kahn, 2017; McKown et al., 2016; Murano et al., 2020; Panayiotou et al., 2019; Ross & Tolan, 2018; Taylor et al., 2017; Weare, 2015). Regarding the promotion of academic achievement, the literature has pointed to positive effects of SEL interventions on reading (Corcoran et al., 2018; Durlak et al., 2011), as well as mathematics and science (Durlak et al., 2011). SEL programs have also been shown to reduce behavioural problems and risk of psychopathology in adulthood (Durlak et al., 2011).

Despite the growing interest around the CASEL and SEL, implementation challenges remain, mainly concerning assessment issues (McKown, 2017). Self-report measures are a valid source of information, since SEL includes emotions and skills that only students have access to (Smith, 2007; Durlak et al., 2015). Limitations associated with this assessment method (e.g., social desirability response bias or and lack of self-awareness; McKown, 2019), seem to be diminished among middle school students (Brackett et al., 2011). The Social Skills Improvement System – Social and Emotional Learning Edition Rating Form – Student (SSIS SEL-RF-S; Gresham & Elliott, 2017) is a self-report scale fully aligned with the five, SEL dimensions previously described. However, it is a lengthy scale, with 46 items, difficult to use for periodic monitoring in school contexts and large samples (Anthony et al., 2020).

To mitigate these difficulties, the Social Skills Improvement System – Social and Emotional Learning Brief Scales – Student Form (SSIS SELb-S; Elliott et al., 2020) was developed. This is a brief scale with 20 items that still describe the five SEL competencies (Anthony et al., 2020). Preliminary studies supported the reliability and validity of the SSIS SELb-S for US students (Elliott et al., 2021; Anthony et al., 2021). Elliott and colleagues (2021) found a Cronbach’s α of 0.90 and test-retest reliability of 0.87 for the composite score of the SSIS SELb-S. Moreover, a reliability level of 0.70 was reported for each SEL domain, which were moderately correlated with each other (0.55 − 0.65).

Although the SSIS SELb-S was created and validated in the American context, its strong technical characteristics, together with its brevity, make it a good candidate for cross-cultural research. Several investigations translated this version and explored its psychometric properties. For example, a study examining changes in adolescents’ SEL skills from three European countries (Latvia, Italy, and Portugal) during the COVID-19 pandemic (Martinsone et al., 2022), found good internal consistency of the composite score in all languages, and acceptable-to-good internal consistency for the five domains (Cronbach’s alfa ranging between 0.53 and 0.84 for Latvia, 0.54 and 0.84 for Italy, and 0.58 and 0.87 for Portugal). Although the reliability of the brief version was comparable to that obtained in previous research (Martinsone et al., 2022), authors cautioned about the less-than-desirable results.

Cavioni et al. (2023) also investigated the validity of the Italian SSIS SELb-S, by analysing a theoretical and statistically factor solution. After eliminating the relationship skills dimension, the authors achieved a 13-item structural model composed of four domains, namely, responsible decision-making, social awareness, self-management, and self-awareness. This structural model was supported by exploratory and confirmatory factor analyses, including measurement invariance tests, which were consistent with other studies (Anthony et al., 2022, 2023), suggesting caution in adopting structural models of the SSIS SELb-S.

Anthony et al. (2022) also provided strong evidence for the convergent and discriminant validity of scores from the translated SSIS SELb-S, for use in several European countries (viz., Croatia, Greece, Italy, Latvia, Portugal, and Romania). They found good levels of measurement invariance across translated versions, which support the use of the SSIS SELb-S for comparative research (Anthony et al., 2022). However, they also found benefits of focusing on the composite score, which is an aggregate score representing the five CASEL domains. Authors argued that this composite score increased the validity and the reliability of the assessment (Anthony et al., 2022), by combining multiple informants (e.g., teachers, parents, self-reports) and different SEL dimensions. In sum, this study indicated that the SSIS SELb-S effectively captured valid, consistent, and reliable information about SEL skills.

As a follow up to this study, Anthony et al. (2023) examined the measurement invariance of the translated SSIS SELb-S from the same six European countries. Results showed that the SEL composite score had a lower level of invariance, being fairly comparable across countries, and, therefore, a global indicator of students’ SEL skills across cultures. However, results indicated that the correlated factors model showed measurement invariance across countries, which was not entirely consistent with CASEL’s five-factor model. The authors explained that students were much less able to distinguish the constructs in SSIS SELb-S, with only a single domain emerging - Social Awareness - although the other four CASEL domains emerged to facilitate the model’s convergence. These results indicated a high level of overlap between some CASEL competence domains, with the domains sometimes more related to each other than within their own domains (Anthony et al., 2023; Frye et al., 2022).

As noted by Anthony et al. (2022), the CASEL framework is best described as an empirically-informed conceptual synthesis, since it did not derive from an empirical technique (e.g., factor analysis). The empirical difficulties that investigations into the CASEL factor structure revealed may indicate that the SSIS SELb may not effectively capture the CASEL constructs as intended (Anthony et al., 2023; Frye et al., 2022). Still, the fact that similar problems emerged across studies may point to issues with the underlying theory rather than with the SSIS SELb-S (Anthony et al., 2023). Perhaps, CASEL domains should not be considered scientific constructs, but rather heuristic conglomerates of distinct SEL domains. They may be useful for practice, but imprecise for guiding scientific research (Frye et al., 2022). For these reasons, some authors proposed that the SEL composite score is a promising indicator of SEL skills in cross-cultural research (Anthony et al., 2023; Frye et al., 2022). Yet, more research seems needed to explain the theoretical specificities of the CASEL framework as well as the validity of original and translated versions of the SSIS SELb-S. As noted before, the different structural models found for SSIS SELb-S puts into question the practical application of the CASEL’s five-factor structure (Cavioni et al., 2023; Anthony et al., 2022, 2023).

To contribute to this research field, the present study aimed to validate the Portuguese version of this scale (hereafter referred to as SSIS SELb-Spt) and examine its validity and reliability in a sample of 5th grade students. We examined the factorial validity, inspected factors’ reliability, and tested correlations between SSIS SELb-Spt factors with external correlates. Since SEL skills also play an important role in students’ academic success, we additionally tested the predictive validity of the SSIS SELb-Spt on students’ academic achievement.

Although CASEL’s 5-factor structure is widely recognised and used, due to its comprehensive and detailed approach, we expected that a 3-factor structure, – intrapersonal skills, interpersonal skills, and responsible decision-making – would fit the data better. By grouping SEL skills into broader factors, the 3-factor structure allows for a more holistic and integrated understanding of SEL and can better capture the interrelationship between different aspects of SEL development. Therefore, the CASEL 3-factor structure may be more suitable in terms of reliability and validity of the measure, while the 5-factor structure may lead to less internal consistency between the factors and affect the validity of the whole structure. We expected that the scale factors would be positively correlated with students’ positive school well-being, as well as negatively correlated with students’ negative school well-being. This hypothesis is based on findings suggesting that students with higher SEL skills recognise and understand their emotions, thoughts, and values, which, in turn, seems associated with greater psychological well-being (CASEL, 2013). We also predicted that the scale factors would be positively correlated with students’ school involvement. Previous research already showed that students with higher SEL skills show empathy, perspective-taking, and appreciation for diversity, which promotes inclusive and supportive school environments, conducive to academic engagement (Durlak et al., 2011). Similarly, we anticipated that the scale factors would be positively correlated with students’ peer acceptance. Students with higher SEL skills seem to have more effective communication, collaboration, and conflict resolution, which promotes positive and healthy relationships between peers (Jones & Bouffard, 2012). Lastly, we also expected that the scale factors would be positively correlated and predict students’ later academic achievement. Since students with higher SEL skills are more likely to achieve goals and maintain concentration and resilience, academic performance may be greater among them (Durlak et al., 2011).

Method

Participants

A priori power analysis using G*Power (1 − β = 0.90, α = 0.05) revealed a required N = 172 to find moderate correlations between SSIS SELb-Spt and external correlates, and N = 61 to detect a medium amount of variance explained in students’ school involvement and academic achievement by SSIS SELb-Spt factors (Brysbaert, 2019). It also revealed a required N = 188 for confirmatory factor analyses (CFA) testing models with five factors, with four indicators. In line with these indications, participants were 200 students attending 5th grade, from two public clusters of schools, one in the North and the other in the Center of Portugal (M = 9.90 years, SD = 0.44; 54% girls). 92.5% of the students were Portuguese, 6.5% Brazilian and 1% Angolan, and all the students were fluent in Portuguese. Since the literature points to mothers’ education years as a good indicator of students’ socio-economic status (Hoff et al., 2002), in our sample this indicator showed an average of 10.90 years (SD = 4.87; Min = 0, Max = 20). All students were authorized by their legal guardians and agreed to participate in the study, which was approved by the Ethics Committee of the Faculty of Psychology and Education Sciences of the University of Porto.

Procedure

In the beginning of the academic year (October 2021), all students were asked to complete an online questionnaire, using google forms, which measured self-report measures, including social and emotional skills, school well-being and involvement, as well as peer acceptance, in this order. The questionnaire was filled out in the computer room of the respective school, in groups that ranged from 8 to 19 students. The experimenter explained the overall procedure and indicated that there were no right or wrong answers. Students read and filled out the questionnaire silently and the individually, which took about 20 min. The students’ grades were also collected at the end of that academic year (July 2022).

Measures

Development of the SSIS SELb-Spt

The original SSIS SELb-S is a 20-item questionnaire with five subscales (α = 0.91; Anthony et al., 2020) – self-awareness, self-management, social awareness, relationship skills, and responsible-decision making –, each with 4 items (α ranged from 0.67 to 0.72 across the subscales; Anthony et al., 2020). Students are asked to indicate how true a set of described situations are, on a 4-point scale (0 = not true; 3 = very true). For each subscale, the average of all items was calculated, so that higher scores reflect greater expertise in the respective skill.

Two Portuguese native speakers, fluent in English, independently translated the original English version of the SSIS SELb-S into Portuguese. After the research group discussed the translated versions, this version was translated back into English by a native English speaker proficient in Portuguese. All items were successfully aligned in meaning with the original text. This final version was then used in the current study.

Self-Report Measures

To measure students’ school well-being, we used the School-Related Well-Being Scales: Affect at School (Kaplan & Maehr, 1999; Portuguese version: Lemos & Coelho, 2010). This is a 7-item self-report questionnaire with two subscales, positive and negative affect, with 3 and 4 items, respectively (α = 0.80 for positive affect and α = 0.78 for negative affect; Lemos & Coelho, 2010). Students are asked to describe how they feel at school, in a 5-point scale (1 = not at all true; 5 = totally true). For each subscale, the average of all items was calculated, so that higher scores reflect a greater positive or negative effect.

To measure students’ school involvement, we used the Involvement subscale of the Self-Reported Engagement (Miserandino, 1996; Portuguese version: Lemos, 2010). This is a self-report questionnaire, where students are asked to describe their effort to remain attentive, focused and engaged in class and school subjects, on a 4-point scale (1 = not true; 4 = very true). The present study used 6 items (out of the 7 scale items) (α = 0.94; Lemos, 2010), excluding the item “When I have a hard question or problem in class, I don’t even try” which, according to the author of the original scale (Miserandino, 1996), has the lowest factor loading. Although the author chose a more conservative approach by not eliminating the item, in the present study, we used the 6 items solution. One item was reverse coded, and the average of all 6 items was calculated, so that higher scores reflect higher school involvement.

Peer Acceptance

To measure students’ peer acceptance, we used a sociometric questionnaire (Asher & Dodge, 1986). This is a hetero-report questionnaire that assesses students’ social and functional position in a group, as well as their sociometric status. Social position is assessed by asking students to name five classmates with whom they would most like to be during the school break, and subsequently, five classmates with whom they would least like to be during the school break. The functional position is assessed by asking students to name five classmates with whom they would most like to work with, and subsequently, five classmates with whom they would least like to work with. Thus, four scores are calculated for each student: peer acceptance and rejection (social position); peer acceptance and rejection (functional position). For each of the four situations, we counted how many times each student was nominated. Each nomination valued 1 point. The relative frequency of the acceptance and rejection points in each of the situations showed us the sociometric position of each student. For this, the total nominations of each student were summed and divided by the number of students in the classroom. In this way, we obtain four measures of functional and social acceptance and rejection.

Academic Achievement

Students’ grades across all school subjects were averaged to obtain a global score of students’ academic achievement. These subjects were grouped into five disciplinary areas – Languages and Social Studies, which includes Portuguese, English, and Portugal History and Geography; Artistic and Technological Education, which includes Musical Education, Technological Education, Visual Education, and Technology, Informatics and Communication; Math and Science, which includes Mathematics, and Natural Sciences; Physical Education; and Citizenship and Development. According to the Portuguese education system, teachers assign grades at the end of each academic year, on a scale ranging from 1 (lowest score) to 5 (highest score).

Data Analyses

We conducted Confirmatory Factor Analyses (CFA) using R (version 4.1.0) (https://www.r-project.org/). After confirming the validity of SSIS SELb-Spt structure, all subsequent analyses were performed with SPSS (version 27).

Confirmatory Factor Analyses

As recommended for models with categorical data (Li, 2016), we fit the CFA models using the robust variant of weighted least squares-mean and variance adjusted (WLSMV) estimator in Lavaan (Rosseel, 2012). Latent variables were scaled by imposing a unit of loading identification constraints (i.e., factors’ variance was constrained to 1), so that all factor loadings could be freely estimated.

We examined the fit indices to evaluate model fit (Kline, 2016): chi-square statistic (χ2) along with χ2/df statistic, confirmatory fit index (CFI), Tucker-Lewis index (TLI), root-mean-square error of approximation (RMSEA), and standardized root mean square residual (SRMR). As indicators of acceptable and good model fit, respectively, we considered, χ2 /df values < 2 and 3, CFI and TLI values ≥ 0.90 and 0.95, RMSEA ≤ 0.06 and 0.10, and SRMR values < 0.06 and 0.09 (Hu & Bentler, 1999; Schermelleh-Engel et al., 2003).

Correlation Analyses

For the model presenting the best fit indices, we examined the items’ factor loadings, item-total correlations, and McDonald’s omega. As recommended by Flora (2020), we computed the omega-higher-order (ωho), which represents the proportion of total-score variance that is due to the higher-order factor. Regarding the reliability coefficient, we followed the rules by George and Mallery (2003). To test correlations with external correlates, we made a stringent test of the factors’ internal structure, by computing the average variance extracted (AVE), with values above 0.50 indicating good convergent validity (Fornell & Larcker, 1981). We additionally calculated the Highest Squared Correlation (HSC), which is the largest squared correlation between a latent construct and any other construct in a structural model, in this case, in the 5-factor model of the SSIS SELb-S. Also, all SSIS SELb-Spt factors were correlated with students’ positive and negative affect, students’ school involvement, as well as with measures of functional and social acceptance and rejection.

Regression Analyses

To examine the degree to which the SSIS SELb-Spt predicted students’ later academic achievement, using one-step regressions analysis, the SSIS SELb-Spt composite score and its factors were independently added.

Results

Confirmatory Factor Analyses

The inspection of descriptive statistics for all SSIS SELb-Spt items (see Table 1) revealed no severe deviations from the normal distribution, as skewness and kurtosis values were below |2.17| and |6.02|, respectively (|𝑠𝑘| < 3; |𝑘𝑢| < 10; Kline, 2016).

Table 1 Descriptive statistics of the 20-item SSIS SELb-Spt (N = 200)
Full size table

Table 2 presents the fit indices of the CFA models applied to the data. We tested a unidimensional model with all 20 items loading on a single SEL factor. The unidimensional model with 20 items revealed an acceptable model fit. The factor loadings ranged from 0.27 to 0.62 (all ps < 0.001). However, item 19 (I try to forgive others when they say “sorry”) presented a factor loading below 0.30, indicating this item had a weak influence on the factor (Hair et al., 2014).

Table 2 Model fit statistics for tested models of the SSIS SELb-Spt (N = 200)
Full size table

Subsequently, we tested a nonhierarchical and a hierarchical 5-factor model with 20 items. The nonhierarchical model, analyses individuals in the dataset independently, disregarding potential hierarchical structures, assuming no shared variance among observations beyond chance (Rasbash & Browne, 2008). In contrast, a hierarchical model, recognizes potential clustering within the data, enabling the estimation of both fixed and random effects, and accommodating the nested nature of the data, facilitating examination of both within- and between-group variation (Raudenbush & Bryk, 2002).

The CFA on the nonhierarchical and hierarchical 5-factor model with 20 items revealed that the covariance matrix of latent variables was not positive definite. These models included impermissible estimates, due to negative estimates of variances or correlation estimates greater than one in absolute value. Due to these Heywood cases, we tested a 3-factor model, namely intrapersonal skills (with the items related to the self-awareness and self-management factors), interpersonal skills (with items related to social awareness and relationship skills factors) and responsible decision-making.

Results on the nonhierarchical 3-factor model with 20 items revealed an acceptable model fit (see Tables 2 and 3). Item-total correlations varied between 0.21 and 0.60, and McDonald’s omega was 0.67 for intrapersonal skills, 0.70 for interpersonal skills, and 0.53 for responsible decision-making. Factor loadings ranged from 0.28 to 0.66 (all ps < 0.001), however, item 19 once again presented a factor loading below 0.30. As detailed in Table 3, AVE was below 0.50 for all factors (range = 0.23 − 0.30) and HSCs were higher than AVE values for all factors. We additionally tested a hierarchical model with SSIS SELb-Spt 3-factors as first-order factors, loading on a single second-order factor. Results on the hierarchical 3-factor model with 20 items presented an acceptable model fit (see Tables 2 and 3). Factor loadings ranged from 0.28 to 0.66 (all ps < 0.001). To analyse the adequacy of using a SEL composite score, we also examined the internal consistency of the higher-order factor of the hierarchical model. The reliability of the composite score with respect to the overall construct of SEL was 0.99. Because these results indicated that both the 20-item non-hierarchical and hierarchical models equally presented a good fit, in the subsequent analyses, we used both the subscales and the composite scores.

Table 3 Factor loading (λ), item-total correlations Range (I-TC), McDonald’s omega (ω), Composite Reliability (CR), average variance extracted (AVE), and Highest Square correlation (HSC) of the SSIS SELb-Spt (N = 200)
Full size table

Correlation Analyses

The associations between SSIS SELb-Spt factors and external correlates are presented in Table 4. We found that SSIS SELb-Spt factors were positively correlated with each other (rs range = 0.53-0.58, ps < 0.01), and each of them was also positively correlated with SSIS SELb-Spt composite score (rs range = 0.83-0.85, ps < 0.01).

Table 4 Correlations between SSIS SELb-Spt factors and the self-report measures of affect, school involvement and peer acceptance and rejection (N = 200)
Full size table

Overall, we found that SSIS SELb-Spt factors were positively related to students’ positive affect (rs range = 0.27 to 0.38, ps < 0.01), and were negatively related to students’ negative affect (rs range = − 0.23 to − 0.35, ps < 0.01). In addition, we observed positive and negative significant correlations between the SSIS SELb-Spt composite score and positive (r = .37, p < .001) and negative affect (r = − .33; p < .001). The two positive and negative affect dimensions were negatively related to each other (r = − .29, p < .01).

Moreover, SSIS SELb-Spt factors and composite score were positively related to students’ school involvement (rs range = 0.48 to 0.68, ps < 0.01). We also found that students’ school involvement was positively and negatively related to students’ positive (r = .21, p < .01) and negative affect (r = − .34, p < .01), respectively.

Concerning the correlations between SSIS SELb-Spt and peer measures, we found that neither the three separate factors nor the composite scores were related to peer acceptance and rejection (social). However, the opposite was found regarding peer acceptance (functional), which was positively related to SSIS SELb-Spt factors (rs range = 0.14 to 0.18, ps < 0.05), as well as to SSIS SELb-Spt composite score (r = .19, p < .01). Moreover, SSIS SELb-Spt composite score was negatively related to peer rejection (functional) (r = − .19, p < .01), as well as to two SSIS SELb-Spt factors, namely, interpersonal skills (r = − .16, p < .05) and responsible decision-making (r = − .19, p < .01). Students’ functional rejection was not related to SSIS intrapersonal skills. It is also worth noting that peer acceptance, both social and functional, were positively related to each other (r = .75, p < .01), as well as peer rejection (social and functional) (r = .88, p < .01). We also found that peer acceptance and rejection (social) were negatively related to each other (r = − .28, p < .01), as well as peer acceptance and rejection (functional) (r = − .40, p < .01). As expected, a negative relationship was found between peer acceptance (social) and peer rejection (functional) (r = − .29, p < .01), as well as between peer rejection (social) and peer acceptance (functional) (r = − .35, p < .01). In addition, as anticipated, peer rejection (functional) was shown to be positively related to students’ negative affect (r = .21, p < .01).

Regression Analyses

Table 5 presents the parameter estimates for the regression model predicting students’ academic achievement. The SSIS SELb-Spt composite score was a significant predictor of students’ global academic achievement, in which responsible decision-making (b = 0.21) was the only predictor factor of students’ global academic achievement. Similar results were found for students’ Artistic and Technological Education achievement, showing that SSIS SELb-Spt composite score was a significant predictor of this disciplinary area, as well as the responsible decision-making factor (b = 0.14). The SSIS SELb-Spt composite score was also a significant predictor of students’ Citizenship and Development achievement, with the intrapersonal skills factor (b = 0.20) as the unique significant predictor of this disciplinary area. In contrast, SSIS SELb-Spt composite score was not a significant predictor of Languages and Social Studies achievement; however, the intrapersonal (b = 0.21) and interpersonal (b = − 0.20) skills factors were significant predictors of this disciplinary area. The SSIS SELb-Spt composite scores, as well as its individual factors, were not significant predictors of students’ achievement in the disciplinary areas of Math and Science, as well as Physical Education (see Table 5).

Table 5 Parameter estimates for the regression models predicting academic achievement
Full size table

Discussion

The present study aimed to develop a Portuguese version of the SSIS SELb-S scale (Elliott et al., 2020), and test its validity and reliability in a sample of 5th graders. Overall, our results seem to corroborate and extend previous evidence using the SSIS SELb-S. The analyses showed that, in the Portuguese sample, the nonhierarchical model with 5 subscales did not work due to Heywood cases, as previously found by Anthony at al. (2021) in a study with English-speaking children. These results seem to be due to the strong relationships between some of the subscales, which is a characteristic of the CASEL’s SEL model (Gresham et al., 2020; Panayiotou et al., 2019).

Given the low fit of the 5-factor solution, we performed a 3-factor analysis of the SSIS SELb-Spt. Consistent with Anthony et al. (2022), we used the dimensions of intrapersonal skills (including the self-awareness and self-management dimensions), interpersonal skills (including the social awareness and relationship skills dimensions) and responsible decision-making. Results showed that both the hierarchical and nonhierarchical 3-factor models achieved acceptable fits. Moreover, correlations between factors ranged from moderate to high, indicating the measurement of related but distinct SEL competencies.

However, in both analyses, item 19, which belonged to the relationship skills dimension, showed unacceptable loading. Although we maintained the item 19, a similar behaviour of the relationship skills dimension was found in the SSIS SELb-S Italian version (Cavioni et al., 2023), which led the authors to eliminate this item and dimension. Thus, the differences in SSIS SELb-S scores between countries may not be due to real differences in SEL competences between countries, but rather to differences in the way the SSIS SELb-S items reflect the constructs or, to fundamental differences in the constructs reflected by the SSIS SELb-S in the different countries (Anthony et al., 2023). Thus, there may be cultural issues associated with the observed variation in the behavioural expression of SSIS SELb-S. For this reason, future research should carefully examine this cultural variation (Anthony et al., 2022).

Overall, we found good reliability for the instrument, except for the responsible decision-making subscale. This suggests that the respective items were not strongly interrelated, and probably not measuring the same underlying construct in a consistent manner. This result raises concerns about the extent to which the scores derived from that subscale provide consistent or accurate representations of the decision-making construct. Furthermore, the examination of the AVE and its comparison with HSCs, yielded less than satisfactory outcomes. The AVE for all SSIS SELb-Spt subscales was below 0.50 and lower than HSCs. These suboptimal results could be attributed to the young age of the participants and, to some extent, the overlapping nature of social and emotional dimensions. In future students, it is imperative to conduct more examinations on SSIS SELb-Spt and investigate whether issues related to the responsible decision-making subscale are specific to this particular sample or if modifications to the items are warranted.

Relations between SSIS SELb-Spt and external correlates were in the expected direction. We found that the SSIS SELb-Spt composite score and the three factors were positively associated with students’ adaptive measures, such as positive affect, school involvement and peer acceptance (functional). We also found that the SSIS SELb-Spt composite score and the three factors were negatively associated with students’ maladaptive indicators, such as negative affect, and peer rejection (functional). In other words, students who had better intrapersonal and interpersonal skills, as well as more responsible decision-making, reported higher positive affect, school involvement, and peer acceptance (functional) as well as lower negative affect. Likewise, students who had lower interpersonal skills and responsible decision-making reported higher peer rejection (functional).

As expected, and pointed out by Taylor et al. (2017), results regarding students’ positive and negative school affect suggest that those with better social and emotional skills have greater positive well-being and lower negative well-being. Concerning students’ school involvement, our findings indicate that students with strong SEL skills are also more likely to participate in classroom activities and engage in learning, which was already defended by Jones and Kahn (2017). These results highlight how social and emotional skills set the stage for students to engage in learning activities and adjust to the demands of classroom-related social tasks (Blair & Raver, 2015), thereby enhancing their experience of school connectedness (Panayiotou et al., 2019).

The relationship between SEL and peer acceptance depended upon the position (social vs. function) considered. Supporting theoretical and empirically data, the SSIS SELb-Spt composite score and its factors were significantly correlated with peer acceptance and rejection in functional but not social contexts. These results are not entirely consistent with the literature. It has been reported that social and emotional skills facilitate the initiation and maintenance of positive social relationships, which contribute to peer acceptance (Gresham et al., 2020; McKown et al., 2016; Murano et al., 2020). However, there is a gap in the literature concerning the influence of social and emotional skills on the separate positions of social and functional peer acceptance. On the one hand, the positive correlation between SSIS SELb-Spt and peer acceptance (functional) is consistent with the literature, showing that children with higher levels of social and emotional skills are more accepted by peers for academic/school work (Denham et al., 1990; Mostow et al., 2002).

On the other hand, the non-significant correlation between SSIS SELb-Spt and peer acceptance (social) diverges from what was expected. Prior research suggested that social and emotional skills allow students to respond and adjust their behavior to social cues, which boosts peers’ acceptance (Boor-Klip et al., 2017; MacCann et al., 2020). Conversely, the results of this study suggest that instead of SEL skills, other perhaps playing-related skills, may be more relevant to choose with whom to socialize. Though a minimum level of social and emotional skills may be need, a set of more instrumental skills for playground activities may be subsequently considered as peer criteria for socialization in playgrounds, at least in this age group. Findings of this study should however be interpreted with caution. Future research seems necessary to examine the differential link between SSIS SELb-Spt and the two positions (social and functional) of students’ peer acceptance.

To assess SSIS SELb-Spt predictive validity, we examined the degree to which its composite and subscale scores predicted students’ later academic achievement. In line with past findings, our study showed that social and emotional skills had a significant predictive value for students’ global academic performance (Durlak et al., 2011; Jones et al., 2017). The responsible decision-making factor also had a predictive value for students’ global academic achievement. Interestingly, findings indicated that the specific disciplinary areas were differently predicted by SEL skills. The scores in the Artistic and Technological Education disciplinary area were predicted by both the composite score and the responsible decision-making factor. These results may be due to the more practical, independent, and autonomous demands of these school subjects. Similarly, grades in the Citizenship and Development disciplinary area were predicted by both the composite score and the intrapersonal factor. This can be explained by the focus on values and personal identity that characterizes this subject area.

Whereas previous studies showed a positive effect of SEL interventions on reading school subjects (Corcoran et al., 2018; Durlak et al., 2011), we found no link between scores in the Languages and Social Studies disciplinary area and the SSIS SELb-Spt composite score. However, we did find intrapersonal and interpersonal skills to predict students’ grades in the Languages and Social Studies disciplinary area. This can be explained by the fact that this subject includes the teaching of oral expression through conversation with others, which involves the mobilization of conversational strategies, as well as collaboration and leadership.

Results also showed that the SSIS SELb-Spt composite score and its factors showed no predictive value in Physical Education as well as in Mathematics and Science disciplinary areas. Although prior results indicated that SEL interventions generally produce positive effects on math and science subjects, these effects seem to be of low practical value, mainly for the science subject (Durlak et al., 2011). Indeed, according to the literature, high-quality randomized studies with larger samples have found very effect sizes for these links (Durlak et al., 2011). Further research is needed to replicate these results and understand the underlying mechanisms responsible for the differential predictive power of SEL skills on specific school areas achievement.

Implications for Practice

Based on the importance of SEL skills in children’s lives, the CASEL framework was developed with the goal of positively influencing the advancement of SEL-focused programs in schools. An essential part of these programs is the universal screening of students’ SEL skills. This study advanced a Portuguese version of a useful and brief measure to map SEL skills, facilitating their use in Portuguese schools and promoting more efficient and effective socioemotional-related actions in these contexts. However, our findings are still preliminary and further exploration is necessary. The version validated in this study, which proved to be a robust instrument, may contribute to this process.

Limitations and Future Directions

When interpreting current findings, some limitations should be kept in mind. First, although we included two schools of considerable size and from two different regions of the country, given the influence that educational and school contexts have on SEL skills, it will be important to expand our sample in future studies. In addition, since our sample included only 5th grade students, future research should attempt to replicate these findings in more diverse samples.

Second, although the present study used a two-time point assessment to examine the link between SEL skills and later academic achievement, no causal inferences can be made. It would be important to conduct randomized longitudinal intervention studies, to examine causal links between SSIS SELb-Spt factors and academic outcomes. Future studies could also help to better understand some of the present findings, particularly concerning the social and functional dimensions of students’ peer acceptance and rejection, as well as the role of SEL skills in students’ achievement across different school subjects.

Third, besides school grades, all variables were gauged with self-report methods, which may have resulted in common-method variance (i.e., the variance observed in the data may be due to the assessment method rather than the actual constructs). Because this issue may compromise the study’s findings, the validity of the SSIS SELb-Spt should be tested in the future using, for example, multi-trait-multimethod approaches.

Fourth, we did not administer other SEL measures and did not test for temporal stability. Thus, current evidence on the SSIS SELb-Spt should be limited to group-level interpretation or intervention planning, as recommended by SSIS SELb-S authors (Elliott et al., 2020). Future research should gather more evidence on the psychometric properties of this scale, as it would be relevant to examine its sensitivity to detect intervention effects.

Fifth, given the lack of other SEL instruments in Portuguese we were not able to assess convergent validity. Given its relevant contribution to an instrument’s validity, we hope future research work to develop further SEL measures in Portuguese and examine their relationship with the SSIS SELb-Spt.

Finally, measurement invariance across gender was not tested, which is crucial to understand if the instrument’s characteristics (factor structure, as well as item parameters) are equivalent between different groups. Therefore, additional studies should conduct these analyses to ensure that there are no systematic deficiencies in the scale that may lead to a differential interpretation of scores by gender subgroups.

Despite the above limitations, it is worth noting that this study is the first to provide validity and reliability evidence on the SSIS SELb-Spt, which is an important tool for developing and evaluating evidence-based SEL interventions in schools.