Introduction

Mental health is defined by the World Health Organization (WHO) as:

“a state of well-being in which the individual realizes his or her own abilities, can cope with the normal stresses of life, can work productively and fruitfully, and is able to make a contribution to his or her community” (WHO, 2004, p.12).

Pupils are specific targets of mental health promotion (Raval et al., 2019) especially because early prevention of mental-ill health problems is found to act as a potential “psychological immunisation” for mental health (Wright, 2015). Several symptoms of adult mental-ill health begin before 14 years of age (Kessler et al., 2007) making it crucial to act with prevention strategies before this age. Furthermore, childhood mental illness negatively affects the normal process of development in all areas (Raval et al., 2019), for instance affecting pupils’ well-being and quality of life (Gaspar et al., 2019) and the quality of learning and academic achievement (Leschied et al., 2018). A meta-analysis from 2020 found a global prevalence, worldwide, of common mental health disorders in adolescents of 25.0% and 31.0%, using the General Health Questionnaire cut-off point of 4 and 3, respectively (Silva et al., 2020). In the context of the COVID-19 pandemic, adolescents presented higher rates of anxiety and depression (Jones et al., 2021). Data from Portugal with a representative sample of schools recently showed that 51.1% of adolescents reported low levels of well-being (Francisco et al., 2023), and a recent comparative survey revealed lower levels of well-being for adolescents since the COVID-19 pandemic (Gaspar et al., 2022).

Screening is seen as an approach that can support in early identification of risk and contribute to early intervention (Silva et al., 2020). Within the school setting, currently, it is proposed for school-based well-being interventions to use the Multi-Tiered System of Supports (MTSS) approach across fundamental areas of mental health (Fabiano & Evans, 2019). MTSS “is a proactive and preventative framework that integrates data and instruction to maximize student achievement and support students' social, emotional, and behavior needs from a strengths-based perspective” (Center on Multi-tiered System of Support, 2022). It is a whole-school approach delivered through three tiers of support, each with increased intensity (universal, selective and indicated), according to a baseline universal screening for identification of pupils’ needs. There is also continuous progress monitoring for decision-making on evidence-based interventions using the schools’ resources/adequately connecting to external services (Hoon et al., 2018; Rodgers, 2019). Other essential components of MTSS are collaboration among school community, orientation for prevention and promotion strategies, supportive leadership and professional capacity (Hoon et al., 2018).

School-based universal screening is a vital component of MTSS, being where all the problem-solving process of MTSS starts: screen, respond, monitor and improve (Center on Multi-tiered System of Support, 2022). Acknowledging the dual-factor model of mental health (Suldo & Shaffer, 2008), a complete mental health (CMH) screening approach is suggested to incorporate mental-ill health symptoms and risk, as well as psychological strengths dimensions (Moore et al., 2019). This can be done by gathering data on all pupils usually through self-, teacher and/or parent reports (von der Embse et al., 2017). For schools to conduct CMH screening, the use of tools which are adapted and validated for their context is required (Allen et al., 2019). To the authors’ knowledge, no study yet exists adapting specific CMH screening tools to be embedded in an MTSS approach for the Portuguese context. This is important because as previously mentioned there are well-being concerns in schools, and so, interventions and supports must be put in place. As MTSS advocates, these supports are more effectively implemented if preceded by a universal screening. Furthermore, universal screening versus other measures for well-being needs identification (e.g. teacher referrals) allows for the possibility of all pupils being identified and reduces possible missed pupils that other traditional methods can miss. A study compared pupils identified by the school and pupils identified by universal mental health screening and found that more pupils were identified through the universal screening process—schools in that study were only serving about 10% of pupils and the screening identified 26% of pupils in need of support (Splett et al., 2018). Moreover, universal screening provides schools with an overview and characterisation of the complete mental health of the school, both strengths and weaknesses, allowing schools to use the strengths when doing interventions for the areas that need support.

Schools’ engagement in the universal screening process is critical for its effectiveness (Moore et al., 2022). Hence, schools must understand the variables being measured and see relevance in them. Recent research has proposed screening efforts for pupils’ psychological well-being to focus on school-specific subjective well-being (Renshaw et al., 2015). School-subjective well-being refers to the subjective satisfaction with domain-specific experiences, in this case, school experiences (Huebner et al., 2014). Hossain and colleagues (2023) analysed the different perspectives on pupil well-being that have been used in the literature, and found some common core elements in the various definitions: positive emotion (e.g. joy) in the school environment and pupils’ appraisal and evaluation of their school experience. These are aligned with the two main approaches to well-being that exist in the literature: hedonic and eudaemonic. Considering the school context, hedonic school well-being relates to satisfaction with school and having a positive affect (e.g. joy) in this environment, whereas eudaemonic pertains to functioning well at school (e.g. school engagement) (Hossain et al., 2023).

Traditionally, assessment of subjective well-being was conducted by collecting domain-general isolated indicators of well-being, such as happiness or life satisfaction (Ruggeri et al., 2020). Yet, recent research has been pointing to a multidimensional approach (Ruggeri et al., 2020), measuring domain-specific indicators. Hence, school-based universal screening of subjective well-being should collect school-related dimensions of subjective well-being (Renshaw & Bolognino, 2016; Renshaw et al., 2015), which will also increase the perceived utility of the measures by schools. Furthermore, a study with college students showed that participants’ school-specific well-being indicators were stronger predictors of academic achievement than the domain-general well-being measure (Renshaw & Bolognino, 2016), reinforcing the importance of collecting domain-specific indicators of school well-being. Houssain and colleagues (2023) found eight domains of pupils’ well-being in the literature: positive emotion, (lack of) negative emotion, relationships, engagement, accomplishment, purpose at school, intrapersonal/internal factors and contextual/external factors.

Renshaw and colleagues (2015) developed a tool, the Student Subjective Wellbeing Questionnaire (SSWQ), which aims to provide schools with a measure of school-specific subjective well-being that can be used alongside mental health risk screening measures and performance-based measures. This contributes to an approach to gaining a comprehensive understanding of the complete mental health status of individual pupils but also provides possibilities to analyse mental health variables at pupil cohort, class and school level. Moreover, the SSWQ was conceptualised and constructed to be used within an MTSS approach, allowing for a brief screening with usable information for interventions and support delivery. To develop SSWQ, the authors did a literature review and found 16 distinct school-specific/generalised to school settings subjective well-being subconstructs: connectedness; gratitude; optimism; zest; meaningful participation; prosocial behaviour; persistence; peer relationships; teacher–student relationships; self-efficacy; goal orientation; educational purpose; curiosity; love of learning; and creativity. Given the screening nature of the tool the authors intended to create, a qualitative theme analysis was performed and the findings suggested that the different subconstructs related to school-specific relationships (e.g. connectedness), educational performance (e.g. self-efficacy), affective experiences (e.g. zest, optimism) and orientations towards pupils’ school experience (e.g. goal orientation). Then, the authors used face validity and selected one subconstruct that seemed to better represent each of the four thematic domains (Renshaw et al., 2015). In the end, the SSWQ measures four specific components of school-related subjective well-being, which were derived from the literature and data analyses: Joy of Learning, Academic Efficacy, Educational Purpose and School Connectedness. The sum of these provides a unified score for subjective well-being (Renshaw et al., 2015). According to the authors, Joy of Learning corresponds to “experiencing positive emotions and cognitions when engaged in academic tasks” (p. 538). An example of an item that measures this dimension is: “I enjoy working on class projects and assignments”. Academic Efficacy means “appraising one’s academic behaviours as effectively meeting environmental demands” (p. 538), and example of an item is: “I get good grades in my classes”. Educational Purpose is defined as “appraising school and academic tasks as important and meaningful” (p. 538). An example of an item that measures this is: “I believe things I learn at school will help me in my life”. The last subscale is School Connectedness which represents “feeling cared for by and relating well to others at school” (p.538), and an example of an item is: “I feel like I belong at my school”.

The psychometric properties of the SSWQ were first assessed in a sample of 1,002 pupils in the USA attending middle school (corresponds to approximately 11 to 13 years old, Grades 6–8). The sample was randomly divided into two subsamples, and both EFA and CFA were conducted. The overall scale (i.e. subjective well-being latent factor) and each four subscales had robust item loadings and good internal consistency reliability. Measurement invariance was assessed, and no lack of invariance across gender was found. Moreover, convergent validity was found with measures of school prosociality and academic perseverance (Renshaw et al., 2015). A replication study was conducted by Renshaw (2015) with a sample of 438 adolescents in Grades 6 and 7 which confirmed the initial study’s latent factor structure, the distribution of the four subscales and good internal reliability, being a positive replication of the technical adequacy of SSWQ. Renshaw and Chenier (2018) then administered the SSWQ to 335 pupils from Grades 5 to 8 and confirmed the SSWQ’s higher-order measurement model (Renshaw & Chenier, 2018).

At present, two cultural adaptations from SSWQ exist in the literature: one in Turkey (Renshaw & Arslan, 2016) and another in Poland (Zadworna et al., 2023). In Turkey, using a sample of 548 adolescents attending Grades 9 to 12 (between 14 and 19 years old), the study showed that the Turkish version of the SSWQ presented a sound latent measurement model, with convergent validity with school-specific well-being constructs. In Poland, the sample was composed of 818 pupils from Grades 5–8 with a mean age of 13.04 years. Results supported both the first-order and second-order latent measurement models with high internal consistency. This Polish version of the SSWQ showed good temporal stability, convergent validity with a quality-of-life measure and divergent validity with an anxiety measure. Hence, other cultural adaptations of this tool support the robustness of the psychometric properties of this tool, making it a sound questionnaire to be used when assessing school-related subjective well-being.

The present study focuses on the screening of the positive dimension of CMH screening to facilitate effective universal screening in schools. This study aimed to adapt and validate a Portuguese version of the SSWQ (SSWQ-PG) and analyse its psychometric properties. The following specific objectives were determined for this study: i) to translate and adapt the SSWQ original version of the SSWQ to create a Portuguese version; ii) to determine the construct validity of the scale using Exploratory Factor Analyses (EFA) and Confirmatory Factor Analyses (CFA); iii) to assess the reliability of the scale in terms of its internal consistency; and iv) to determine the internal validity of the scale by analysing the convergent and divergent validity of the scale with related variables.

Method

Participants

A total of 942 pupils participated in this study from 10 different schools in Portugal across diverse regions: North of Portugal (four schools; 37.8% of participants), Centre (two schools; 11.2% of participants), Lisbon and surrounding areas (three schools; 33.9% of participants), and Island of Madeira (one school; 17.2% of participants). All schools were public except for one that was a private school in Lisbon. The average socio-economic status of the pupils of each school varied from low middle class to middle class. Two schools were designated as TEIP (Educational Territories of Priority Intervention) schools.Footnote 1 Four schools were in urban areas, three schools in rural areas and two schools in semi-urban areas.

All pupils were in the 3rd cycleFootnote 2: 42.8% in the 7th grade; 27.9% in the 8th grade and 29.2% in the 9th grade. Regarding their age, 21.7% were 12 years; 31.7% were 13; 27.4% were 14 years, and 15.9% were 15 years. Only 3% were 16 years old (28 cases) and three cases were 17 years old, possibly pupils that were repeating years of education due to previous difficulties or events. Pupils were asked to provide their gender in a free box. There was a balanced percentage of pupils identified as male (50.8%) and female (47.9%), three cases identified as non-binary, two pupils chose to not respond, and three cases wrote non-identified.

Measures

Student Subjective Wellbeing Questionnaire (SSWQ) (Renshaw et al., 2015). This questionnaire has 16 items and measures school-specific subjective well-being. It is comprised of four subscales: Academic Efficacy (AE) (e.g. “I am a successful student”); Educational Purpose (EP) (e.g. “I feel like the things I do at school are important”; Joy of Learning (e.g. “I feel happy when I am working and learning at school”); and School Connectedness (SC) (e.g. “I can really be myself at this school”). The original version is in English and was developed in the USA. The questionnaire uses a four-point response scale: 1 = almost never, 2 = rarely, 3 = sometimes and 4 = almost always. All items are positively written, and no reverse coding is needed. Previous studies (Renshaw, 2015; Renshaw et al., 2015) have consistently found a psychometrically sound higher-order factor structure, with four first-order factors, indicators of an underlying second-order factor (i.e. pupils' subjective well-being). Each subscale and the global scale had adequate internal reliability (Cronbach alpha of ≥ 0.70). Within this study, a translation and adaptation of the original English version was conducted to adapt this tool to the Portuguese context.

The translation of the tool to Portuguese followed the International Test Commission Guidelines (International Test Commission, 2017). The following steps were taken: (1) a member of the team who is bilingual in Portuguese and English did a first translation of the tool to Portuguese (European); (2) another member of the team also bilingual reviewed this translation to create a unified preliminary version; (3) then an external member of the team bilingual in both languages did a back-translation to English, and no changes were deemed necessary to the preliminary version; and (4) this version was then analysed by two teachers, two school psychologists and three adolescents aged between 12 and 13 years from public schools. It was asked for them to assess the language clarity of the items, their cultural appropriateness and representativeness of well-being in school. As a last step, the result of the previous step was reviewed by two external experts on the topic of well-being and socio-emotional development.Footnote 3 Finally, a final Portuguese version was reached and used to collect data within this study. During the back-translation process, no changes were deemed necessary to be implemented in the preliminary translation to Portuguese. The panel of pupils, school psychologists and teachers suggested changing items 8 and 12 to “I do, without difficulty (…)”. The original item is written “I do good work at school” and the suggested change was seen by the panel and the two experts as better for the Portuguese language. Another suggestion was on item 16 “I get good grades in my classes”. This item was changed to “I get good grades in my evaluations”, since in the Portuguese educational system, pupils only receive grades in the evaluation moments, and no grades are provided during class tasks.

The results section will describe the results of the adaptation of this tool to Portuguese. The same factor structure was expected as in previous studies: a second-order latent model with four subscales, with adequate model fit and reliability.

Children’s Worlds: The international survey of children’s well-being (The Children’s Worlds project). From this questionnaire, six items related to satisfaction with school were selected to be used for analyses of convergent validity. The Portuguese version was used in previous studies (Costa, 2015; Costa & Dias, 2016). On a scale from 0 = Not at all satisfied until 10 = Totally satisfied, pupils were asked how satisfied they were with the following things in their life: other children in their class; their grades at school; their school experience; their life as a student; the things they have learned; and their relationship with teachers. No reverse coding is needed. Although a Portuguese version of these items existed, no psychometric properties in Portugal were found, so EFA with the entire sample (N = 942) of this pool of items was conducted in this study. The Kaiser–Meyer–Olkin test (KMO = 0.860) and Bartlett’s sphericity test (χ2 = 1856,577, df = 15, p < 0.0001) showed adequacy of the data. A one-factor solution was extracted based on eigenvalues above 1, explaining 50.10% of the variance of the factor. Factor loadings ranged from 0.498 to 0.878. Cronbach’s alpha was 0.845.

The group identification scale (GIS; Sani et al., 2015). This questionnaire includes four items which were adapted to the present study aiming at examining the student’s level of social identification with their friends' group at school: I feel a bond with my [group of friends at school]; I feel similar to the other members of my [group of friends at school]; and I have a sense of belonging to my [group of friends at school]; I have a lot in common with the members of my [group of friends at school]. This questionnaire was used to analyse convergent validity. No reverse coding is needed as greater scores on all items indicate greater identification. The GIS measures two fundamental aspects of social identification: feeling of group belonging and feeling of sharing similarities with other members of the group. The scale of response to the items ranges from 1 = Totally disagree to 7 = Totally agree. In this study, EFA and CFA were conducted for this scale since it has not previously been used in Portugal. Related to EFA, conducted with a subsample (n = 453), the Kaiser–Meyer–Olkin test (KMO = 0.823) and Bartlett’s sphericity test (χ2 = 1090,468, df = 6, p < 0.0001) showed adequacy of the data. A one-factor solution was extracted based on eigenvalues above 1, explaining 68.90% of the variance of the factor. Factor loadings ranged from 0.572 to 0.805. Related to CFA, conducted with a second subsample (n = 489), a good model fit was found: χ2 = 15,568, df = 7,784, p < 0.0001, CFI = 0.989, GFI = 0.984, RMSEA = 0.120 (90%CI: 0.069 to 0.178). Cronbach’s alpha was 0.900.

Social Support (McNamara et al., 2021). Social support was measured with eight items adapted in the present study to collect information on students’ perception of social support from teachers and from non-teaching staff separately: “I get the emotional support I need from my teachers/non-teaching staff”; “I get the help I need from my teachers/non-teaching staff”; “I get the resources I need from my teachers/non-teaching staff”; and “I get the advice I need from my teachers/non-teaching staff”. Participants were asked their agreement with each item on a scale of 1 (not at all) to 5 (completely). This questionnaire was used to analyse convergent validity, as well. No reverse coding is needed as high scores on all items indicate greater support. In this study, EFA and CFA were conducted for this scale since it has not previously been validated for teachers and non-teaching staff and used in Portugal. Related to EFA, conducted with half the sample (n = 453), the Kaiser–Meyer–Olkin test (KMO = 0.876) and Bartlett’s sphericity test (χ2 = 2254,626, df = 28, p < 0.0001) showed adequacy of the data. A two-factor solution was extracted based on eigenvalues above 1, explaining 67,78% of the cumulative variance: one factor for the items of social support from teachers and another factor for the social support from non-teaching staff. Factor loadings ranged from 0.590 to 0.744. Related to CFA, conducted with the other half of the sample (n = 489), a good model fit was found for a correlated two-factor model: χ2 = 108,471, df = 5,165, p < 0.0001, CFI = 0.961, GFI = 0.943, RMSEA = 0.094 (90%CI: 0.077 to 0.112). Cronbach’s alpha was 0.863 for the teachers’ items and 0.905 for the non-teaching staff’s items.

BPM-Y: Brief Problem Monitor-Youth version 11–18 years old (Achenbach et al., 2011). This is a behaviour rating instrument for monitoring adolescents' psychological functioning by gathering their self-report behaviours representing mental health problems. This questionnaire was used in this study to analyse for discriminant validity. The items were adapted for self-completion by the authors from existing measures by Achenbach and Rescorla (2001). Each item is rated 0 = not true, 1 = somewhat true or 2 = very true. The BPM includes items for rating problems defined as relating to attention (ATT), internalising (INT) and externalising (EXT). These are summed to get the Total Problems (TOT) score. No reverse coding is needed. The Portuguese version used in this study was translated and adapted within the project MIPA-Mobile (Projeto MIPA-Mobile, 2015). In this adaptation, the following Cronbach’s alphas were found for a normative sample: ATT: 0.715; INT: 0.675; EXT: 0.637; and TOT: 0.794. In this study’s sample, the Cronbach’s alpha were: ATT: 0.760; INT: 0.818; EXT: 0.772; and TOT: 0.807.

Procedure

This study was approved by the Ethical Committee for Health Universidade Católica Portuguesa (no. 85) and approved by the Portuguese Ministry of Education (no. 0128800009). Several schools with 3rd cycle provision were invited to participate in the study, activating different networks of the researchers in this study and using the snowball method to find more schools possibly interested in participating. The research was explained to the school board and was then approved (or not) by the school’s Pedagogical Council. Once schools agreed to participate, opt-in informed consent forms from the responsible parent/person of pupils were requested and signed. An average of a two-week timeframe was provided, and then, pupils whose parents provided positive consent were invited to participate by responding to the questionnaires within the school contexts. Participants responded to the questionnaire between May and June 2023. Both paper–pencil and online Qualtrics forms were used, depending on the school conditions to collect data through computer/tablet/phone. Anonymity was ensured, by not asking participants for their names, and also associating their responses with a school code. Data collected through paper–pencil were inserted in SPSS, whereas data collected with Qualtrics was exported directly to SPSS. Confidentiality and anonymity of responses were ensured using protected data storage and identification codes in place of identifying information. Codes were used to identify responses from each school for analysis and to provide an opportunity for data extraction in the case of withdrawal.

Statistical Analysis

All analyses were conducted using IBM SPSS version 28 and IBM SPSS Amos Statistics 28. The statistical analysis plan was in this order: descriptive analysis; EFA with subsample one; CFA with subsample 2; reliability testing with both subsamples and the whole sample; measure invariance with gender and school year with subsample 2; and discriminant and convergent validity with the whole sample.

First, descriptive analysis of the main variables and item analysis were performed considering the recommendations from Kline (2016) and Marôco (2010). Then, considering this was the first time the selected measures were used in the Portuguese context, and a cross-validation approach was applied (Floyd & Widaman, 1995) to ensure robustness and reliability of the factor structure, randomly dividing all data set into two independent samples: subsample 1 (n = 453) for EFA and subsample 2 for CFA (n = 489). Student t-tests for independent samples were conducted considering the gender and age of pupils, and no significant differences were found between the two subsamples. The reliability analysis was conducted both within each subsample and using the entire sample. The listwise method was chosen in all analyses to manage missing data, whereas to perform CFA, all participants with at least one item missing (i.e. did not respond to at least one item in the SSWQ), were excluded from the analyses (subsample 2 for CFA ended with 463 participants; after this deletion, other 48 cases were removed from subsample 2 because of being characterised as a multivariate outlier using the “Mahalanobis distance” procedure) (Marôco, 2010).

Using subsample 1, EFA was conducted taking into consideration the Kaiser–Meyer–Olkin (KMO) test and Barlett’s sphericity test. Factors were retained based on eigenvalues above 1.0 (Kaiser, 1960), parallel analysis (Hayton et al., 2004) and scree test results (Cattell, 1966). Subsample 2 was then used for CFA, selecting the maximum likelihood estimation method, following analyses used by the authors of this measure (Renshaw et al., 2015). Both four first-order latent factors and then a second-order latent factor were used. Data–model fit statistics were used to understand the validity of the model (Mueller & Hancock, 2008), and the following indices were considered for the goodness of data–model fit: Tucker–Lewis Index (TLI), goodness-of-fit index (GFI), comparative fit index (CFI) and root mean square error of approximation (RMSEA). The cut-off values interpreted as indicating an adequate model fit were: for TLI and CFI between 0.90 and 0.95 (Marôco, 2010); for RMSEA with a 90% confidence interval between 0.05 and 0.010 (Marôco, 2010); and for GFI a value equal or above 0.90 (Jackson et al., 2009). Factor loadings above 0.50 were interpreted as appropriate loadings for factor validation (Marôco, 2010).

After CFA step was finished, reliability (internal consistency) was studied using Cronbach’s alpha and McDonald’s omega on the entire sample (N = 942) and each subsample, considering values equal to or above 0.70 as acceptable reliability (Kalkbrenner, 2023). Only using subsample 2 and data retrieved from CFA, Composite Reliability (CR) was also assessed using Raykov (1997) estimation and the calculator from Colwell (2016) for each latent factor.

Measurement invariance of the final SSWQ-PG model was tested for gender (female, male) and school year (7th, 8th and 9th grades). This was done considering a configural invariance, metric invariance and scalar invariance. In the first, factor loadings, item intercepts and error variances were unconstrained across groups. In the second, factor loadings were constrained to be equal, with the previous, unconstrained model. And in the last, equivalence constraints were imposed on the intercepts. As mentioned above, fit indices were considered for TLI, CFI and RMSEA. The Chi-square was considered to analyse the significance of the differences between groups considering pΔχ2 < 0.05 (Marôco, 2010) as non-variant measure.

Convergent and divergent validity analyses were conducted with the entire sample. Pearson’s correlation between each factor of the SSWQ-PG version and the Group Identification scale, the items of school satisfaction and social support, interpreted according to Akoglu (2018). Using the complete sample, divergent validity was assessed with correlations between each subscale and total scale of the SSWQ-PG version, and BPM-Y subscales and total scale – negative correlations between well-being and internalising/externalising symptoms were expected due to the negative association between well-being and mental health problems (Bartels et al., 2013). For convergent validity, a positive correlation was expected with satisfaction with school as school satisfaction is typically associated with health (Horanicova et al., 2022). A positive correlation was also expected between social identification and social support as studies have shown the relationship between social identification (Miller et al., 2015; Sani et al., 2012) and school-related social support (Tian et al., 2016) with well-being.

Results

Item Analysis

Descriptive analysis of each item Table 1 showed that missing responses per item varied between 3 and 14 participants, all items have a minimum of 1 and maximum of 4, and no extreme values of skewness and kurtosis were found. All items showed good item–total correlation, being above 0.4, ranging between 0.420 and 0.628. Frequencies of responses per response scale option, between almost never and almost always, show no option having 80% or more of the total responses per item, there are no ceiling effects as well. Table 2 presents the response frequencies per scale. Correlations between all items were significant except for correlations between item 1 and items 14 and 16, which was not considered an issue since these items are part of different subscales. All correlations were below 0.70. No multicollinearity issues were detected, and all items had VIF numbers between 1.451 and 2.142.

Table 1 Descriptive analysis of SSWQ items (entire sample)
Full size table
Table 2 Frequencies of responses on SSWQ items (entire sample)
Full size table

Internal Structure

EFA

EFA was conducted using subsample 1 (n = 453). Maximum likelihood extraction with Promax rotation was conducted, following analyses used by the authors of this measure. The Kaiser–Meyer–Olkin test (KMO = 0.849) and Bartlett’s sphericity test (χ2 = 2480,577, df = 120, p < 0.0001) showed adequacy of the data. A four-factor solution was extracted, as in the original version, based on eigenvalues above 1, explaining 29.48%, 8.56%, 6.72% and 5.52% of the variance of each factor, with 50.3% of cumulative variance explained. Both the scree plot and parallel analysis supported a four-factor solution. All items showed adequate communality values, ranging between 0.333 and 0.689 (Table 3). All factor loadings were considered acceptable (Table 3). Item 13 showed low cross-loadings (0.396 in factor 1; 0.316 in factor 2), and due to this, it was deleted from the Portuguese SSWQ version leaving a * item scale. Item 3 showed high factor loading in a different factor than it is associated with in the original SSWQ version: in this analysis, item 3 (“I feel like the things I do at school are important”) was highly related to Joy of Learning subscale, instead of Educational Purpose. Given the factor loading clearly putting this item in this subscale, it was decided to move item 3 to the Joy of Learning subscale.

Table 3 SSWQ factor loadings and communalities
Full size table

CFA

To confirm the structure of the SSWQ-Portuguese version, a CFA with a maximum likelihood estimator was performed using subsample 2 (n = 463). Results from the EFA supported the following decisions when designing the model: Educational Purpose was the factor with higher variance explained, and so, the regression weight was fixed between Student Subjective Wellbeing and this factor; in each factor, the fixed regression weight was set on the item with higher factor loading in the EFA; the scale consisted of a total of 15 items as item 13 was deleted; and item 3 was placed on the factor Joy of Learning, instead of Educational Purpose.

For the first-order latent factors, goodness-of-fit indices indicated that the data fitted the proposed four-factor model: χ2 = 271,0966, df = 84, p = 0.000, CFI = 0.923, TLI = 0.904, GFI = 0.921, RMSEA = 0.073 (90%CI: 0.064 to 0.083). All values were adequate, according to previously established cut-off values mentioned in the Method section. The second-order model (see Fig. 1) showed similar results to the first-order model and therefore was the preferred model: χ2 = 280,290, df = 86, p = 0.000, CFI = 0.920, TLI = 0.903, GFI = 0.915, RMSEA = 0.074 (90%CI: 0.064 to 0.084). Standardised regression weights were all above 0.50 in all items, ranging from 0.56 to 0.84.

Fig. 1
figure 1

CFA second-order latent factor model path diagram for the SSWQ-PG version

Full size image

Internal Consistency

Internal consistency was evaluated in both subsamples and the whole sample (Table 4). These results are presented as well in Table 4. All results show good reliability of each factor (all above 0.70). The model was finally kept (Fig. 1 model).

Table 4 Internal consistency analysis of the SSWQ-Portuguese version in both subsamples and in the entire sample
Full size table

Measurement Invariance

In Table 5, the fit indices are summarised for the different models examined in the invariance routine for gender (female, male) and school year (7th, 8th and 9th grades). Independent multigroup analysis were conducted for gender and school year. As can be seen, full gender and school year invariance (i.e. configural, metric and scalar) could be considered attained (pΔχ2 > 0.05).

Table 5 Fit indices of measurement of the SSWQ-PG for gender and school year
Full size table

Convergent and Divergent Validity

Table 6 presents results for convergent and divergent analyses, showing overall significant weak negative correlations between the SSWQ-PG scale and BPM-Y scale, except for between the internalising subscale and Educational Purpose factor of SSWQ-PG. Conversely, all correlations with the items referring to satisfaction with school, to social identification and to social support, are significant and positive, meaning that SSWQ-PG showed convergent validity with these measures. With the satisfaction with school, correlations were moderate, and correlations with social identification and social support varied between weak to moderate.

Table 6 Convergent and Divergent validity of SSWQ-PG version
Full size table

Discussion

This study presents the first adaptation to Portuguese and validation of the Student Subjective Wellbeing Questionnaire in Portugal, for adolescents. This is relevant to the practice of universal well-being screening in schools with adolescents as it provides professionals with a tool that can be used to collect data on positive well-being functioning. After conducting both EFA and CFA of the SSWQ-PG with different subsamples, the results confirmed adequate psychometric properties of the SSWQ-PG version, with adequate fit of the tool (excluding item 13) producing a 15-item measure with high internal consistency and support for its validity.

EFA showed a four-factor solution, which was then tested in CFA and showed adequate fit; however, item deletion and factor assignment based on psychometric performance in both EFA and CFA and theoretical assumptions were deemed appropriate.

EFA results suggested the removal of one item (“I feel happy when I am working and learning at school”). This item showed low cross-loadings, and due to this, it was not considered in the subsequent analyses. This result is inconsistent with previous validations of the SSWQ to Turkey (Renshaw & Arslan, 2016) and Poland (Zadworna et al., 2023); hence, it might be specific to the Portuguese language. The exclusion of item 13 was, hence, not supported by previous literature but based on its anomalous psychometric functioning.

Results also showed that item 3 (“I feel like the things I do at school are important”) saturated onto Joy of Learning subscale, instead of Educational Purpose. Given the factor loading clearly putting this item in this subscale, with a factor loading ≥ 0.50, it was decided to move item 3 to Joy of Learning subscale. This decision is supported by conceptualising and findings in intrinsic motivation literature. The perceptions/cognitions captured in this item can be viewed as associated with cognitions surrounding the value given to academic tasks (Pintrich & De Groot, 2003). Higher levels of perceived value are known to increase motivation, particularly intrinsic motivation. So, if pupils like more the things they are currently doing at school, this would increase their sense of enjoyment which is related to intrinsic motivation (Ryan & Deci, 2020), and feeling enjoyment relates to the Joy of Learning subscale.

Similar to previous research, a second-order latent factor model yielded adequate fit in this study, being Student Subjective Wellbeing as the higher-order latent factor (e.g. Renshaw et al., 2015; Zadworna et al., 2023). This shows that each factor score and the total score for subjective well-being can be used when applying SSWQ-PG in research and practice.

Related to the internal consistency of this questionnaire, excellent results were found in each subsample and for the whole sample. The internal consistency was similar or slightly higher to other SSWQ versions (Renshaw & Arslan, 2016; Renshaw et al., 2015; Zadworna et al., 2023).

Full measurement invariance was attained when comparing male and female participants and participants from 7th, 8th and 9th grades. We can also assume that the structure of the measure is invariant for age since different ages are present in each school year.

As expected, the SSWQ-PG positively correlated with the measures used to examine convergent validity (satisfaction with school, social support and friends group identification), although correlations with the social identification and social support questionnaires varied from weak to moderate. This could be explained because SSWQ measure is focused on school-related subjective well-being in school, and only one subscale, School Connectedness, measures social dimensions related to well-being. This subscale did have a moderate correlation with both social identification and social support questionnaires.

The results of this validation process are consistent with results from previous cultural adaptations of the SSWQ tool (Renshaw & Arslan, 2016; Zadworna et al., 2023). However, the item deletion and the change of one item to another dimension was not suggested in these previous studies, which emphasises the potential language-specific or cultural influences on item functioning, reinforcing the importance of cross-validation such as the one performed in this study.

The SSWQ-PG showed negative correlations with BPM-Y. However, no correlation was found between the Educational Purpose subscale and the Internalising subscale of BPM-Y, which may be explained by the fact that some internalising experiences, especially anxiety at a lower level, within a general sample as this one (non-clinical sample) may contribute to higher Educational Purpose and academic success. A previous meta-analysis supports this idea since it showed that anxiety has a small-level positive effect on pupil academic achievement (Erzen, 2017).

This study has several limitations. One limitation may be related to the sample, as there are no schools (hence pupils) to represent every region in Portugal. However, this study has the representation of pupils from public and private schools, different areas in Portugal including one of the islands, and urban, semi-urban and rural schools. Also, this study only focused on pupils in the 3rd cycle, not ensuring a validated version for other school years in Portugal, however, this cycle refers to the beginning of adolescence when universal screening is particularly important to contribute not only to support pupils with lower well-being, but as well to prevent it during a key moment in adolescence. Last, this study may have a mono-method and mono-source biases as data was only collected from pupils’ self-report only through questionnaires, hence a portion of the observed associations between constructs is due merely to the fact that pupils reported on their behaviour using questionnaires for all constructs.

This study also has several strengths. It is the first study to provide a validated version of the SSWQ for use with adolescents in Portugal. The internal structure of the SSWQ-PG was analysed by both EFA and CFA using two subsamples considering the large number of participants in the whole sample. The translation process was done following the International Test Commission Guidelines (International Test Commission, 2017), and a panel of not only adolescents but also school psychologists and teachers who are experienced was considered.

In conclusion, the present study shows adequate psychometric properties of the Portuguese version of the SSWQ. This validation has practical implications as this means the SSWQ-PG can be used in Portuguese schools with pupils in the 3rd cycle of their schooling for screening school-related subjective well-being, alongside other measures to screen for the complete mental health of pupils. Using this tool, schools will be able to inform interventions to support mental health and academic success. Furthermore, this measure relates to measures of social identification, social support and satisfaction with school. This validation also contributes to research, providing a tool that can be used in studies related to school relationships and well-being in Portugal. Further explorations of the psychometric properties of the SSWQ-PG are also possible to be conducted and suggested to continue assessing the technical adequacy of this tool in Portugal, contributing to practice.