Introduction

Throughout the past two decades, educational, social, and political interest in certain protective factors against social, emotional, and behavioural problems among pupils in compulsory education has increased considerably. The accumulation of scientific evidence on these factors’ capacity to explain pupils’ individual differences in performance or well-being in different spheres of life has contributed to this (Oberle et al. 2016). Among these protective factors, Social and Emotional Learning (SEL) stands out, which is conceived as the process through which pupils acquire and effectively use the knowledge, skills, and attitudes necessary to develop healthy identities, manage their emotions, set and achieve positive personal and collective goals, feel and show empathy for others, establish and maintain positive and supportive interpersonal relationships, manage interpersonal situations constructively, and make responsible and caring decisions (Collaborative for Academic, Social, and Emotional Learning, 2021; Durlak et al, 2011; Jagers et al. 2019; Mahoney et al. 2020; Payton et al. 2008; Taylor et al. 2017; Weissberg et al. 2015).

From this SEL framework, in this paper, we intend to assess the effectiveness of a programme aimed at improving social and emotional competencies among pupils registered in Basic Vocational Education and Training (VET) from disadvantaged contexts. SEL entails the ability to combine behaviour, cognition, and affect (Mahoney et al. 2020), providing learners with the precise tools to successfully deal with any situation in their daily lives, which is essential to improving their learning, performance, and satisfaction (Oberle et al. 2016; Organisation for Economic Co-operation and Development, 2021; Weissberg et al. 2015). Indeed, the accumulated scientific evidence surrounding this essential component for pupils’ personal, social, and emotional development has greatly encouraged the implementation and evaluation of various intervention programmes and practices aimed at creating safe and supportive learning environments in which to promote the five core social and emotional competencies: self-awareness, social awareness, self-control, relationship skills, and responsible decision-making (Collaborative for Academic, Social, and Emotional Learning, 2021; Jagers et al. 2019; Mahoney et al. 2020; National Commission on Social, Emotional, and Academic Development, 2019).

Self-awareness is the ability that enables one to identify one’s own emotions, thoughts, and values and how they affect behaviour, including assessing strengths and limitations with confidence, enthusiasm, and a growth mindset, while social awareness is the ability required to empathise with and understand the perspective of others from diverse contexts and cultures, including defending one’s own ideas without belittling others (Collaborative for Academic, Social, and Emotional Learning, 2021). Self-control is conceived as the ability to successfully regulate one’s own thoughts, emotions, and behaviours in different situations, including setting school goals and working towards their attainment with self-discipline and self-motivation, employing planning and organisational strategies (Collaborative for Academic, Social, and Emotional Learning, 2021). On the other hand, relationship skills concern the ability to establish and maintain appropriate relationships with other people, including effective communication, active listening, and cooperation with others (Collaborative for Academic, Social, and Emotional Learning, 2021). Finally, responsible decision-making is the ability required to make positive and constructive choices based on personal behaviour and relationships with others based on ethical standards, safety, and social norms, including the assessment of the consequences of actions and personal well-being and that of others, in addition to identifying problems, proposing solutions, and implementing actions that contribute to improving the immediate environment (Collaborative for Academic, Social, and Emotional Learning, 2021).

There are numerous studies that have established causal relationships between intervention measures based on SEL and certain improvements at the socioemotional, behavioural, and school level of the participating pupils, regardless of their socio-demographic and educational profile (i.e., racial, ethnic, and socioeconomic background, from different educational levels and environments, with and without emotional and behavioural problems), as confirmed by the first systematic reviews developed to determine the impact of these programmes and practices (i.e., Diekstra, 2008; Durlak et al. 2010, 2011; Payton et al. 2008; Sklad et al. 2012). Moreover, the results of meta-analytic reviews that have subsequently been conducted for the same purpose are along the same lines (i.e., Corcoran et al. 2018; Jagers et al. 2015; Murano et al. 2020; Sabey et al. 2017; Taylor et al. 2017; Wigelsworth et al. 2016; Yang et al. 2019), showing that this type of intervention measures has repeatedly demonstrated its ability to improve pupils’ social and emotional skills, self-perceptions, attitudes towards others, commitment and bond with the school institution, prosocial behaviour and school performance, generating a decrease in their emotional, behavioural and substance abuse difficulties and even effects on other members of the educational community (e.g., higher rates of effectiveness and achievement in teachers’ planning). However, most of these studies and the studies included in these systematic reviews have been developed in Anglo-Saxon contexts.

These results have contributed to the fact that interventions based on SEL are among the most successful development programmes and practices, which has led to their rapid and extensive diversification and incorporation into educational institutions and classrooms around the world (Wigelsworth et al. 2016). Nevertheless, Fernandez et al. (2021) carried out a systematic review aimed at synthesising the main characteristics and evidence on the effectiveness of SEL programmes in Ibero-American contexts, and they concluded that the quality of the scientific evidence that allows for the establishment of causal relationships between participation in SEL programmes and the improvement of school and socio-emotional outcomes is relatively low, due to several factors. First, many SEL programmes do not consider the key indicators that guarantee their success (Durlak et al. 2010, 2011; Mahoney et al. 2020), while other SEL programmes do not incorporate evaluation measures or provide exclusively participation and/or satisfaction results among their participants (Fernández et al. 2021). Second, most of the empirical data comes from studies using qualitative, pre-experimental, or ex post facto research designs, which limits the power of the evidence available on their effectiveness (Fernández et al. 2021). Third, many programmes show several limitations in the size of the samples or the intergroup comparability (Fernández et al. 2021). Therefore, there is an unquestionable need to increase the quality of scientific production in the design, implementation, and evaluation of programmes and practices based on SEL in Ibero-American contexts, which in turn will make it possible to establish national agendas that promote its adoption throughout the educational systems, integrating it into existing educational priorities (Fernández et al. 2021).

In this sense, this study aims to contribute to expanding the empirical evidence available on the power and validation of a SEL causal and logical model to generate improvements in the social and emotional competencies of compulsory education pupils at risk of social exclusion in Ibero-American contexts. The programme reported here consists of a psycho-pedagogical intervention based on the SEL model, which incorporates in its logic model a series of elements and features aimed at repairing the weaknesses and shortcomings pointed out above. Therefore, this SEL programme combines school instruction with the explicit instruction of social and emotional competencies, based on a sequenced, step-by-step training approach that emphasises active forms of learning, concentrating specific time and attention on skills training, and in which goals are clearly defined (i.e., a sequenced, active, focused, and explicit training), although it also incorporates the active role of participants and the training of teachers responsible for implementation, as well as the collaboration and synergies between classrooms, families, and communities (Durlak et al. 2010, 2011; Fernandez et al. 2021; Mahoney et al. 2020). Furthermore, a quasi-experimental evaluation methodological design with a non-equivalent control group enhanced with statistical control techniques and a system of monitoring and continuous improvement was established (Durlak et al. 2010, 2011; Fernandez et al. 2021; Mahoney et al. 2020).

More concretely, this SEL programme is a transformative educational intervention, an integral model (i.e., mixes formal and non-formal, social and community, as well as personal and professional elements), and an innovative action model that aims to change the life trajectory of young people in Basic VET from disadvantaged backgrounds (Tomillo Foundation, 2022). Basic VET programmes are an alternative option offered by the Spanish education system to pupils who have not completed lower secondary school to stay in education and training (European Centre for the Development of Vocational Training, 2022), in order to reduce the early school dropout rate, which in 2021 stood at around 13%. However, 35% of the pupils enrolled in these programmes drop out before completing the first year, with the Basic VET being the educational stage where the greatest school failure in the entire system is concentrated (Ministerio de Educación y Formación Profesional, 2021). For this reason, the SEL programme focuses its efforts on improving the competency profile of these young people from a social, emotional, and labour perspective, which will allow them to advance in school, have more opportunities for social and labour insertion, and have a life plan in which they can make informed decisions about their future (Tomillo Foundation, 2022).

Therefore, the purpose of this research was to test the effectiveness of a SEL programme to improve social and emotional competencies (i.e., self-awareness, social awareness, self-control, relationship skills, and responsible decision-making) among Basic VET programme pupils at risk of leaving education from disadvantaged contexts. To this end, the following hypotheses were established: (1) the pupils experimental group will improve in statistically significant terms their average scores obtained in social and emotional competencies in post-test with respect to pre-test as a result of their participation in the programme, while in the case of the pupils control group no such differences will be observed; and (2) as a result of the programme, there will be a statistically significant improvement in the pupil’s experimental group’s social and emotional competencies at the end of the school year, reflected in the Social and Emotional Learning Scale (Fernández et al. 2022), as compared to that of the pupil’s control group.

Method

Sample

The sample of this study consisted of 110 first-year pupils of three Basic VET programmes taught in an educational centre in the southern district of the city of Madrid (Spain). This sample was divided into two equivalent groups: experimental (n = 55) and control (n = 55) groups. Therefore, the sample size exceeded the result of calculating the minimum sample size needed to evaluate the program, both overall (n = 102) and by group (n = 51).

The first one consisted of the experimental group with 55 pupils, 14 women and 41 men, with an average age of 15.98 years (SD = 0.83, age range: 15–17 years) and a distribution by Basic VET programme of 19 pupils studying electricity and electronics, 17 studying computer and communications, and 19 studying administrative services. The second one consisted of the control group with 55 pupils, including 14 women and 41 men, with an average age of 15.87 years (SD = 0.84, age range: 15–17 years), and a distribution by Basic VET programme of 20 pupils studying electricity and electronics, 15 studying computer and communications, and 20 studying administrative services (see Tables 1 and 2 for more detail).

Table 1 Experimental and Control Groups Equivalence: Socio-demographic Variables.
Full size table
Table 2 Experimental and Control Groups Equivalence: Educational Variables.
Full size table

The sample selection procedure was based on a non-probabilistic convenience sampling technique (Gertler et al. 2017) and involved three actions. First, a secondary school centre and the Basic VET programmes offered there were chosen as the field of study due to their availability; the objectives of the research project were aligned with the needs and goals of the educational centre and the Basic VET programmes; and there were limited financial, material, and human resources available (April 2021). Second, two group information sessions with the institutional authorities, teachers, and educators from the selected educational centre and the Basic VET programmes were carried out, in which they were informed about the characteristics of the intervention while they were invited to participate in the programme (April 2021). Third, once the required institutional permissions were granted, teachers, educators, and pupils from the two classroom groups of each of the three Basic VET programmes offered at the school were enrolled in the programme after signing the programme school and family agreements (September 2021). However, out of the 126 pupils who were initially enrolled in the programme by teachers and educators, two pupils from the experimental group did not attend at least 80% of the classes during the implementation of the programme, while 5 pupils from the experimental group and 9 pupils from the control group dropped out of the Basic VET programme, so the final sample was made up of 110 pupils. We compared the characteristics of the 16 pupils who dropped out of the study with the characteristics of the final sample pupils, and we did not find significant results, so this is suggestive evidence that attrition bias is likely low.

Instruments

The Social and Emotional Learning Scale (Fernández et al. 2022) is a Likert-type scale consisting of 30 items with four response levels (1 = never or rarely, up to 4 = almost always or always), grouped into five subscales (i.e., self-awareness, social awareness, self-control, relationship skills, and responsible decision-making). This scale was used because it is one of the few instruments available to assess the different competency areas of the social and emotional learning model (Collaborative for Academic, Social, and Emotional Learning, 2021; Jagers et al. 2019; Mahoney et al, 2020; National Commission on Social, Emotional, and Academic Development, 2019) in the Spanish adolescent population, as well as for presenting adequate reliability (Cronbach’s alpha between 0.70 and 0.84, and McDonald’s omega between 0.71 and 0.84, with composite reliability and variance extracted indices above 0.77 and 0.67, respectively, in the different areas of social-emotional competence) and validity (tests with excellent goodness-of-fit indices that confirm its internal structure and predictive validity on school performance and life satisfaction) (Fernández et al. 2022).

The Participation Questionnaire is an ad hoc self-report made up of 28 items with different response alternatives, aimed at collecting the following information relevant to the control variables of the participating pupils: (a) socio-demographic: age, sex, marital status, nationality, immigration background, employment status, the highest level of education successfully completed by a member of your household, source of study funding, and socioeconomic status; and (b) educational: educational centre, Basic VET programme, programme year and modules or subjects registered, specific educational support needs, number of previous retakes, change of previous studies, and dropout of previous studies.

Design and procedure

The methodological design adopted was a quasi-experimental design with a non-equivalent control group enhanced with statistical control techniques (Gertler et al. 2017). The study was carried out in accordance with the Ethical Committee of the University of Granada (1736/CEIH/2020) and the Declaration of Helsinki (World Medical Association, 2013).

Once the sample was selected, the experimental and control groups were randomly formed, respecting the natural grouping of the classrooms. Thus, each classroom group in each Basic VET programme was randomly assigned to either the experimental group or the control group (September 2021). However, it was confirmed that both groups were equivalent in some observable characteristics that could influence the results and/or be affected by the intervention (i.e., control variables) (Gertler et al. 2017) (September 2021). In this sense, some of the control variables showed the same value in both groups (i.e., marital status, employment status, educational centre, programme year and modules or subjects register, year of access to the programme, change of previous studies), others showed the same proportion in both groups (i.e., sex, socioeconomic status, specific educational support needs), while for the rest of the variables, no significant statistical differences were found between both groups (i.e., age, nationality, immigration background, the highest level of education successfully completed by a member of your household, source of study funding, Basic VET programme, number of previous retakes, dropout of previous studies, social and emotional competencies) (see Tables 1, 2 and 3).

Table 3 Intragroup (Pre-Test vs. Post-Test) Comparisons of Pupils’ Social and Emotional Competencies.
Full size table

The SEL programme was implemented in the school during the whole scholar year, from September 2021 to June 2022, although the training of the teachers and educators responsible for its execution and the piloting of the programme were carried out during the previous school year (2020–2021). This intervention, as the specialised literature established (Durlak et al. 2010, 2011; Fernández et al. 2021; Mahoney et al. 2020), focused on the explicit instruction of social and emotional competencies based on sequenced, active, focused, and explicit training and was effectively integrated with school instruction. To this end, the Basic VET curriculum was developed in six transversal projects, in which the contents of different modules were addressed through active methodologies (i.e., project-based learning, with a service-learning approach and cooperative learning), with their corresponding evaluative processes, using tools aimed at favouring reflection, metacognition, and conscious learning of pupils (i.e., learning portfolios, learning diaries, rubrics, and self-assessment and co-assessment questionnaires). These projects were carried out by a team of teachers and educators in a special classroom, a technical, open, and flexible space for group and individual work for pupils, but also involved the development of different meaningful experiences in which pupils had the opportunity to actively consolidate learning from a different scenario. Among these meaningful experiences were the “disruptive learning pills”, which involved work sessions in artistic-musical, environmental, sports, or technological scenarios. Other noteworthy experiences were the outings outside the educational centre to favour experiential learning (e.g., visits to professional work centres, learning visits to different cities, etc.), the technical sessions given by company professionals (e.g., workshops and training sessions given by professionals of their professional profile in order to broaden their exposure to the professional world and learn about different realities of various national and international companies), and the field exploration and research sessions (e.g., exploration of the environment and collection of evidence and analysis in an experiential manner).

Likewise, this SEL programme incorporated in its logic model the implementation of other activities in parallel with the development of the transversal projects mentioned above. In this sense, a mentoring programme between pupils and young people from the educational centre was implemented to facilitate access to school and the personal adjustment of pupils. Moreover, individualised tutoring was carried out to detect potential situations of school failure as well as to provide socio-educational support to pupils and families (e.g., interviews), in addition, to support for personal development and the personalisation of pupils’ learning itineraries (e.g., welcome dynamics to leave behind emotional states that do not favour learning and to determine itineraries according to vocation and interests). Also, vocational orientation sessions were performed to help pupils learn about their strengths and passions so they could make informed decisions about their future. Additionally, internships in a professional environment were developed. These internships were designed in a personalised way for the pupils, identifying the opportunities for the development of technical skills and social and emotional competencies that each company can offer in order to make the pupil-company assignment based on the needs of the company and the pupil’s technical and competency development. Finally, professional orientation sessions were implemented to jointly assess access to the labour market, job offers, up-skilling or re-skilling programmes, invitations to participate in events, etc.

In parallel to the intervention plan, the monitoring plan to guarantee the fidelity of the SEL programme (i.e., actions aimed at monitoring possible deviations of the programme) was implemented (Fernández et al. 2019). In this regard, three group follow-up sessions were held between programme staff and teachers and educators (end of the first, second, and third trimesters of the school year), and two group follow-up sessions were held with the participating pupils (i.e., end of the first and second trimesters of the school year). These sessions were mainly devoted to identifying difficulties in the development of the programme’s projects and activities, proposing and implementing appropriate solutions, and assessing their overall participation in the programme. Finally, with the evaluation plan of the results, pre-test (September 2021) and post-test (June 2022) measurements were taken of the dependent variables to test for statistically and/or educationally significant effects (Fernández et al. 2019).

Data analysis

Firstly, the power analysis and the minimum sample size needed were calculated considering the expected effect size (0.50), the associated probability (0.05), and the desired levels of statistical power (0.80) (Soper, 2022).

Subsequently, following the recommendations of Tabachnick and Fidell (2019), the initial analysis consisted of checking linearity, atypical, missing, and influential cases (Mahalanobis distance), and data distribution (i.e., asymmetry and univariate and multivariate kurtosis: Kolmogorov-Smirnov test and Mardia’s coefficients of multivariate skewness and kurtosis), as well as the descriptive analysis of the control and dependent variables.

In addition, thirdly, in order to test the equivalence of the experimental and control groups on the control variables, non-parametric analyses were performed: (a) Mann–Whitney U test: age and social and emotional competencies; and (b) Pearson’s chi-squared: age, nationality, immigration background, the highest level of education successfully completed by a member of your household, source of study funding, Basic VET programme, number of previous retakes, and dropout of previous studies.

Fourth, to determine the effectiveness of the programme, the different hypotheses were analysed through the Wilcoxon z test (hypothesis 1) and the Mann–Whitney U test (hypothesis 2). In addition, Cohen’s d value was calculated, while the family-wise error rate, resulting from the multiple comparison problem, was controlled with Bonferroni correction.

Statistical analyses were carried out using Statistical Package for the Social Sciences 28.0 (IBM Corp., Armonk, NY, USA) and STATA 17 (StataCorp., College Station, TX, USA).

Results

The results of the initial analysis confirmed the absence of atypical, missing, and influential cases, and the Kolmogorov–Smirnov test and Mardia’s coefficients of multivariate skewness and kurtosis did not yield a normal univariate and multivariate distribution of the pupils’ pre-test and post-test average scores on the Social and Emotional Learning Scale (Fernández et al. 2022). Moreover, the descriptive analysis of the dependent variables is shown in Tables 3 and 4, and tests of the hypotheses were conducted using Bonferroni adjusted alpha levels of 0.01 per test (0.05/5).

Table 4 Intergroup (Experimental vs. Control) Comparisons of Pupils’ Social and Emotional Competencies.
Full size table

The results of the pre-test and post-test comparisons on social and emotional competencies (hypothesis 1) do not reveal statistically significant differences in the control group, while in the case of the experimental group, a statistically significant improvement is observed in the post-test with respect to the pre-test in each of the social and emotional competencies (Table 3).

Regarding hypothesis 2, the results of the intergroup comparisons in the pre-test do not show statistically significant differences in any of the social and emotional competencies, unlike in the post-test, in which statistically significant differences in favour of the experimental group are observed in each of the social and emotional competencies (Table 4).

Ultimately, the intergroup comparisons of pupils’ social and emotional competencies in the post-test phase yield medium (between 0.5 and 0.8) and large (0.8 or greater) effect sizes, which can be interpreted as a significant impact of the programme on these competencies. In fact, these results are similar to those of the intragroup (pre-test-post-test) comparisons of the experimental group, with trivial effect sizes (less than 0.15) in the case of the control group.

Discussion and conclusion

The purpose of the study was to verify the type, direction, and magnitude of changes produced by a SEL programme on the social and emotional development of a sample of Basic VET programme pupils at risk of social exclusion, specifically the impact on each of the areas of competence of the SEL model (i.e., self-awareness, social awareness, self-control, relationship skills, and responsible decision-making). Therefore, taking into consideration the results derived from participation in the programme, the following conclusions can be established: (a) Since in the intragroup comparisons statistically significant differences are observed in the experimental group in favour of post-test in the social and emotional competencies, quite contrary to the case of the control group, hypothesis 1 is accepted; and (b) since statistically significant differences are appreciated in the intergroup comparisons in favour of post-test in the social and emotional competencies, not so in the control group, hypothesis 2 is likewise accepted.

Indeed, participation in the SEL programme has generated a positive and statistically significant impact for the pupils in the different indicators of social and emotional development, despite the highly restrictive nature of the Bonferroni correction. Furthermore, if the hypotheses of this study are contrasted using tests that provide answers about their practical significance, as established in the specialised literature (Hattie 2009; Kraft 2020; Ledesma et al. 2008), considering the most demanding guidelines for interpreting the results of analyses complementary to the contrast of means (Cohen, 1988), the magnitude of the effect size achieved in most of the areas of social and emotional competencies has been large, that is, the intergroup differences generated in these areas of competence can be detected by simple observation (Coe 2002), which clearly indicates that they have an important practical relevance (Hattie 2009; Kraft 2020). In fact, effect size values reveal that a hypothetical member of the experimental group, compared to anyone in the control group, can achieve scores in these competency areas above 69% (e.g., relationship skills), with percentages that can even reach above 88% (e.g., self-awareness) (Coe 2002; Kraft 2020).

The results obtained confirm the efficacy of the SEL programme to increase the level of social and emotional competencies of compulsory education pupils at risk of social exclusion, which will possibly facilitate their promotion at school, provide them with more opportunities for social and labour market insertion, and provide them with a life plan in which they can make more informed decisions about their future. Likewise, these results are in line with those of other studies that have been developed with the purpose of establishing causal links between programmes and practise based on the model of SEL and certain improvements in the social and emotional level of their participants, as can be seen in the meta-analytic reviews that have been conducted in the last decade (Corcoran et al. 2018; Durlak et al. 2010, 2011; Jagers et al. 2015; Murano et al. 2020; Sabey et al. 2017; Taylor et al. 2017; Wigelsworth et al. 2016; Yang et al. 2019), and equally favours increasing the quantity and quality of the limited scientific production on the systematic evaluation of this type of interventions at the national level (Fernández et al. 2021).

However, it is necessary to consider certain methodological limitations when interpreting the results obtained in this study, mainly linked to the sampling and design adopted, since a completely randomised sampling and design would have allowed greater control of possible sources of bias, although the requirements and resources of the entity responsible for managing the programme made this impossible. It could also be interesting in future editions to increase the number of covariates or control variables, as the selection of the right covariates is the most challenging issue with these methodological designs for establishing unconfoundedness, along with their reliable measurement. Another aspect to consider in future editions of the programme is to increase the sample size, although it was adjusted to the standard conditions of power and alpha error, but above all to broaden the diversity of Basic VET programmes and programme years involved. Obviously, it is always advisable to change those aspects of the programme that, because of process evaluation, could be improved (e.g., coordination between teachers and educators).

This research contributes to expanding the empirical evidence available on the power and validation of the causal and logical model of the SEL programme to generate improvements in the social and emotional competencies of compulsory education pupils at risk of social exclusion, in addition to providing educational institutions with an efficient educational tool or project that can contribute to change and improve the life trajectory of their pupils. In this sense, taking as a reference the international quality standard in educational interventions (e.g., evidence for ESSA), it can be stated that this SEL programme, both for its methodological rigour and its results, is a clear example of an evidence-based programme (Slavin 2017). However, the limited number of studies with these characteristics and methodological rigour at the national level (Fernández et al. 2021) recommends continuing to support the systematic evaluation of this type of intervention, mainly to accumulate more evidence and improve the impact of these programmes on the social and emotional development of pupils in compulsory education.