Advertisement

The Survey of School Promotion of Emotional and Social Health (SSPESH): A Brief Measure of the Implementation of Whole-School Mental Health Promotion

  • Katherine L. Dix
  • Melissa J. Green
  • Stacy Tzoumakis
  • Kimberlie Dean
  • Felicity Harris
  • Vaughan J. Carr
  • Kristin R. Laurens
Open Access
Original Paper

Abstract

This paper describes the development of a brief school-based instrument designed for use in research and in educational practice. The Survey of School Promotion of Emotional and Social Health differentiates states of high, moderate, and low implementation of whole-school policies and practices that promote the emotional and social health of students. The instrument measures the extent to which a school has implemented policies and practices in four health-promoting domains: (a) creating a positive school community, (b) teaching social and emotional skills, (c) engaging the parent community, and (d) supporting students experiencing mental health difficulties. Responses were gathered via an online survey of Principals in almost 600 Australian primary schools in New South Wales. Preliminary psychometric properties of the instrument, and the development of an implementation index using latent class analysis, are described. The final 13-item version of the survey has broad applicability for use by researchers and evaluators for comparative and multivariate analyses. School leadership may find it useful as a brief tool to guide the identification of target areas for whole-school improvement across the four important health-promoting domains.

Keywords

Whole-school improvement Implementation index Social and emotional wellbeing KidsMatter Psychometrics 

Introduction

Improving the capacity of schools to enhance students’ academic achievement and wellbeing is a central focus of governments in Australia (e.g. Gillard, 2010; Jensen & Sonnemann, 2014) and internationally (e.g. Ainscow and West, 2006; Fullan, 2007; Resnick, 2010; Sammons, 2007). Schools play a key role in developing the social and emotional skills that underpin students’ mental health and wellbeing alongside academic learning (Brackett et al., 2009; Feigenberg, Watts, & Buckner, 2010; Mellin et al., 2016). They are recognised as critical sites for the prevention and early identification of mental health problems, as well as the promotion of wellbeing (Green et al., 2013; Payton et al., 2008; Sklad, Diekstra, Ritter, Ben, & Gravesteijn, 2012; Durlak et al., 2011; Taylor et al., 2017). Importantly, a whole-school approach to promotion of emotional and social health stands to benefit all students, not just those at risk for poor mental health (WHO, 2004). Research indicates that “schools will be most successful in their educational mission when they integrate efforts to promote children’s academic, social, and emotional learning” (Zins et al., 2004) and that “strong bonds between student behaviour, attainment and learning and their social and emotional development” are central (Sammons, 2007). This evidence has been actioned in nation-wide school-based initiatives that address students’ wellbeing and bolster each child’s capacity for effective academic learning (e.g. CASEL, 2008; DCSF, 2009; ENSEC, 2009; KidsMatter, 2009; Benningfield et al., 2015).

The extent to which such initiatives are implemented is critical in effecting change (Durlak and DuPre, 2008; Basch, 2011). Given the complexity of implementing mental health promotion school-wide, some schools do this more readily than others. For example, in Australia, significant positive relationships were found between the extent of implementation of a whole-school mental health initiative and students’ differential mental health outcomes (Slee et al., 2009) and academic performance (Dix, Slee, Lawson, & Keeves, 2012). In order to assess implementation quality in the 96 schools studied, Dix, Slee, Lawson, and Keeves (2012) used latent class analysis to develop an ‘implementation index’. The index categorised schools as high, moderate, or low implementers of the mental health initiative based on a set of criteria (fidelity, dosage, quality of delivery) and found that high implementing schools showed greater academic gains than low implementing schools. Similar results have been reported elsewhere (e.g. Simonsen et al., 2012; Payton et al., 2008; Sklad, Diekstra, Ritter, Ben, & Gravesteijn, 2012; Durlak et al. 2011; Kidger, Araya, Donovan, & Gunnell, 2012; Wells, Barlow, & Stewart‐Brown, 2003; Basch, 2011; Novins, Green, Legha, & Aarons, 2013; Weissberg, Goren, Domitrovich, & Dusenbury, 2013). However, despite the large number of school-based programs that foster positive mental health, there continues to be little evidence available concerning how schools measure the implementation of such programs and the impact that they may have (Adelman & Taylor, 2000; Domitrovich et al., 2008; Durlak and DuPre, 2008). Feigenberg, Watts, and Buckner (2010) suggest that it has been challenging to measure the impact of formal whole-school mental health interventions at the school-level due in part to the lack of assessment tools that measure school-level characteristics (Tseng and Seidman, 2007). In addition to the plethora of formal programs, schools may also be engaged in mental health-promoting activities outside the remit of a formal program, suggesting the need for assessment tools that are independent of the programs.

A number of instruments assessing implementation have focused on the whole-school setting. These include assessments that examined cultural climate and school safety (e.g. Brand, Felner, Shim, Seitsinger, & Dumas, 2003), school-wide positive behaviour (e.g. Horner et al., 2004), school mental health capacity and quality (e.g. Feigenberg, Watts, & Buckner, 2010; Weist et al., 2006; Wyn, Cahill, Holdsworth, Rowling, & Carson, 2000; Graetz et al., 2008), and school health (e.g. Staten et al., 2005). However, while there are toolkits and self-assessment planning guides that schools can use to improve their wellbeing policies and programs (e.g. KidsMatter, 2013; Staten et al., 2005; E4L, 2017), their emphasis is not quantitative and they do not readily enable psychometric assessment, nor allow comparisons between schools, and do not allow school wellbeing promotion to be readily examined as an outcome variable. Accordingly, research and practice stand to benefit from the development of a quantitative assessment tool that describes the extent to which whole-school mental health promotion policies and practices have been implemented. For this reason, we developed the Survey of School Promotion of Emotional and Social Health (SSPESH).

In the development of the SSPESH, we adopted a conceptual approach reflecting protective factors known to influence student wellbeing. This was based on the World Health Organization’s (WHO) Health Promoting Schools (HPS) framework that incorporates: (a) ethos and environment of the school; (b) formal health curriculum; and (c) engagement with families, or communities, or both (Langford et al., 2014). In Australia, the major health-promoting school initiatives, such as KidsMatter and MindMatters, operationalised the HPS framework by separating (c) into two parts to create four components that foster the mental health of every student: (a) positive school community; (b) social and emotional learning for students; (c) parenting support and education; and (d) early intervention for students experiencing mental health difficulties (Graetz et al., 2008; Beyondblue, 2017). This conceptual approach guided our design of the SSPESH measurement tool.

Research Context

This research was conducted as part of the New South Wales Child Development Study (NSW-CDS) in Australia (Carr et al., 2016; Green et al., in press). The NSW-CDS is a multi-agency, intergenerational data linkage study that brings together diverse health, education, welfare, and justice records in a population cohort of approximately 90,000 children and their parents. It aims to identify childhood factors that promote positive mental and physical health, and social and educational outcomes in adolescence and adulthood. It also aims to identify new opportunities for detecting and mitigating early vulnerability for ill-health and other adverse outcomes. In 2015, the investigator team administered online a self-report survey of mental health and wellbeing to a subsample of children in the NSW-CDS cohort (the Middle Childhood Survey; MCS), via 829 primary (elementary) schools in NSW (Laurens et al., 2017). MCS data were obtained from 27,792 children aged approximately 11 years of age during their final year of primary schooling (Year 6 in NSW). In addition, the NSW-CDS surveyed the primary school Principals (or their delegate from the school leadership group, such as a Deputy Principal) regarding the extent to which whole-school mental health promotion activities were implemented in the participating school. In Australia, the Principal is the head of the school and oversees its management and administration. Promoting school improvement is an important part of a Principal’s role, which can be assisted by self-assessment tools that help monitor and identify particular areas for improvement.

The aim of the present study was to develop a brief, school-level instrument with the potential to profile schools and identify characteristics that differentiate high, moderate, and low implementation of whole-school promotion of emotional and social health. This approach has been successfully used in national evaluations of whole-school mental health promotion initiatives in Australia (Dix, Slee, Lawson, & Keeves, 2010; Slee et al., 2012; 2009). It acknowledges that implementing whole-school policy is complex and that schools in diverse contexts accomplish this to varying extents. The diversity of school contexts in Australia arise from systemic and geographical factors (e.g. government and non-government systems, metropolitan and remote location, socio-economic disparity, multiculturalism). A key feature of this approach was to develop a measure of implementation (an ‘implementation index’) that could account for the likelihood that not all schools implement whole-school wellbeing promotion to the same degree. We sought to strengthen capacity to attribute significant differences in implementation and the impact this may have on students. Rather than arbitrarily assigning implementation thresholds (cut-points) to a total survey score, latent class analysis (LCA) was used to profile schools. LCA is a statistical method for identifying subtypes of related cases (latent classes) from multivariate categorical data. The main benefit of using LCA was its non-dependence on assumptions of normality and the establishment of non-arbitrary scoring thresholds (cut-points) for the instrument (Dix, 2009). Accordingly, this paper discusses the development and preliminary validation of the SSPESH and its derived ‘implementation index’, and considers potential uses of the SSPESH in research and practice.

Method

Instrument Development

The SSPESH was based on two earlier measures developed in a broader context to meet the needs of schools participating in the Australian KidsMatter Primary initiative (2013). These precursor tools were designed for and completed only by schools participating in the KidsMatter initiative; though no psychometric analyses have been published on those survey tools, they closely aligned with our requirements for the current study and provided a useful pool of items for consideration. The precursor surveys were developed over a number of years by integrating stakeholder opinion from a team of mental health and education experts with evidence from existing literature on whole-school mental health promotion (KidsMatter, 2017). The 52-item ‘Staff Survey’ and the 31-item ‘School Check-up’ were designed around the four-component KidsMatter framework to assess a school’s capacity to promote mental health and wellbeing across the school community (Graetz et al., 2008). While the original items were written for generalisability across school settings, many of the items were considered to be too conceptually complex and poorly designed.

The goal in developing the SSPESH was to simplify the previous measures and create a brief, generic school-level instrument (not aligned to a specific program or initiative such as KidsMatter) that could assess a school’s capacity to promote social and emotional wellbeing. Importantly, to minimise respondent burden and to maximise response rates by NSW-CDS schools, the instrument had to be shorter than its predecessors and easily completed by school Principals. As the leader of the school, Principals have overall responsibility for monitoring and promoting social and emotional health in their school. Predominantly, these types of school climate and wellbeing surveys have targeted teachers, parents, and students. This can render the information collection process more complex and often ‘too hard’ to undertake within the context of the busy school environment. We sought to design a brief tool that could be completed quickly by school leadership to support an efficient reflective process that informed school improvement policy.

As part of the instrument development process, the lead author (with an academic background in educational measurement and survey design) consulted multiple experts in school-based mental health, school leadership, whole-school improvement, and questionnaire design from three federal agencies (Beyondblue, Australian Psychological Society, and Principals Australia Institute). These consultations guided the selection of items that were thought to align with the four conceptual domains identified in the literature as important agents of change in terms of mental health promotion, prevention and early intervention (e.g. Graetz et al., 2008) and school improvement (e.g. Jensen & Sonnemann, 2014); namely: (a) positive school community, (b) student social emotional learning skills, (d) working with families, and (d) supporting students experiencing mental health difficulties.

While the conceptual elements in the original items, sourced from the ‘Staff Survey’ and ‘School Check-up’, were retained, in consultation with the experts, the wording of each item was modified to gather information about whole-school practices rather than those of individual staff, and to ensure that the questions were not specific to a particular initiative (e.g. KidsMatter). Care was taken to ensure that the survey items used plain English and were readable by educators. This was reflected in a Flesch reading-ease score of 42, equivalent to College level (Flesch, 1980). Nevertheless, the items were complex in nature in order to convey a breadth of related content. For example, the item, Our school has specific policies and practices that promote inclusion and a safe environment, comprised two components (inclusion and a safe environment). While a high implementing school could be expected to have both in place, a school with only one of these aspects in place would be expected to report a lower rating on this item. This approach enabled us to keep the survey brief while maximising the breadth of content, and also lowered the risk of a measurement ceiling effect (i.e. it was more difficult for schools to select the top response option).

In the final stages of measurement design, the item-redevelopment process involved cognitive interviews and a focus group discussion conducted with a convenience sample of six school Principals and school wellbeing coordinators. A cognitive laboratory approach was taken (Leighton, 2017). Participants were guided through a short-list of possible items, to gauge their understanding around the intent of each item, and identify items that were confusing, not relevant, unclear, or too complex. The final items were selected because they were perceived to be readily identifiable and rateable by school leadership. The resultant measure, the SSPESH, included 14 statements that characterised favourable practices within a school, rated on a four-point Likert-type scale of: Not yet in place (scored 0), Introducing (1), Taking hold (2), and Completely in place (3). To comprehensively characterise school performance in wellbeing promotion, the SSPESH was designed to provide an overall summary score as well as domain scores for each of the four components.

The instrument also contained additional questions that were not intended to be used in the calculation of the total SSPESH score, and as such, are not reported in detail in this paper. These questions asked about specific social and emotional learning (SEL) programs and wellbeing frameworks being implemented by schools. School leadership could identify up to five SEL programs from a list of over 100 formally verified programs (Trinder, Roberts, & Cavanagh, 2009). They were asked to indicate when the program commenced, what year levels were targeted, and their perception of how effective the program was. An option to provide other SEL programs not listed was also available.

The SSPESH also asked schools to indicate if they were implementing any whole-school mental health frameworks. This selection included KidsMatter Primary, which provides mental health promotion to primary school students according to the four-component framework that guided the development of the SSPESH (Graetz et al., 2008; Beyondblue, 2017).

Sampling and Procedure

A target sample of 689 NSW primary schools with a Year 6 enrolment were invited to participate in the research; these were a subset (83%) of the 829 schools that had participated in the MCS and indicated willingness to be contacted with an invitation to participate in a Principal/school leadership survey of school-based mental health promotion.

Principals (or their delegate from the school leadership group, such as a Deputy Principal) were invited via email to complete the SSPESH following the conclusion of the MCS administration in October 2015. Participants completed the SSPESH using an online survey (software supported by PeoplePulse; www.peoplepulse.com). A unique survey link was provided to each school. Informed consent was indicated via survey completion. To maximise response rates, participants were given a 2-month period in which to complete the survey. Responses were monitored during the administration period, and system-generated reminder emails were sent to those schools yet to respond, followed by a telephone reminder to the Principal, yielding the participation of 598 schools.

The SSPESH online survey presented the original 14-item instrument as a set of statements and radio-button choices, in matrix format. This was followed by the section capturing school implementation of SEL programs and whole-school wellbeing frameworks, which used drop-downs, check-boxes and open-text response formats. The brief survey was presented on a single page, and no items enforced mandatory responding.

All procedures performed were in accordance with the ethical approvals issued by the University of New South Wales Human Research Ethics Advisory Panel D: Biomedical (Reference number: HC14348) and the New South Wales Department of Education’s State Education Research Approvals Process (SERAP reference number: 2015083).

Statistical Analyses

Analyses were conducted using SPSS Version 22 (IBM Corp, 2013) and Mplus version 5.3 (Muthén & Muthén, 2007).

Structural Analysis

We first examined the distribution of the 14 individual items, and determined inter-item correlations using Spearman’s rank-order correlation to account for any non-normal distribution of items. Reliability was assessed using Cronbach’s alpha coefficient, and the item-total correlations were examined to identify items that appeared not to contribute to the overall construct being measured by the SSPESH.

Exploratory factor analyses (EFA) were then conducted, initially on the full 14-item scale, to examine the instrument’s dimensionality and to identify for removal any item that did not contribute to the overall scale (based on the item-total correlation coefficients and on non-loading of any item with other items). A split-half methodology was adopted for the structural analysis in Mplus, such that the sample was split randomly into two groups of 299 and 298 schools, respectively, with the EFA conducted on one sample, and confirmatory factor analysis (CFA) on the other. EFA was used to explore solutions with one to five factors using Oblimin rotation (Muthén & Muthén 2007). Goodness-of-fit indices used to evaluate model fits included the root mean square error of approximation (RMSEA), the standardised root mean square residual (SRMR), the comparative fit index (CFI), the Tucker Lewis Index (TLI), and the normed Chi-squared statistic (χ2/df). These indices perform better than other indices under non-normal distribution conditions (Fan, Thompson, & Wang, 1999; Hu & Bentler, 1999). Factors were evaluated for non-loading and cross-loading items, using a threshold for loading of 0.3. Next, we examined the factor structure of the resultant instrument using CFA. Like the EFA, model fit was examined using maximum likelihood parameter estimates with standard errors that are robust to non-normality (Hooper, Coughlan, & Mullen, 2008).

Finally, in order to provide a check of the criterion validity of the resulting SSPESH structure, we used the information collected from the schools regarding their engagement with the KidsMatter Primary whole-school initiative. Independent t-tests were conducted on the SSPESH total scale (summed) score, and on the subscale summed scores based on the factors derived from the CFA, comparing scores for schools that indicated that they had engaged, verses those that had not engaged, with KidsMatter Primary.

Latent Class Analysis

LCA was conducted in Mplus to identify profiles (classes) of schools that could be distinguished with respect to the quality of implementation reported (i.e. to identify classes of schools with similar patterns of responses to the SSPESH items). Using the four response options (Not yet in place, Introducing, Taking hold, and Completely in place) for each of the items, LCA models were conducted to determine the optimal number of latent classes signifying varying degrees of implementation. Robust maximum likelihood (MLR) was used to estimate solutions for one to four classes. The number of initial stage random starts was set at 50 with the 15 best solutions retained for final stage of the optimisations. To obtain the best solution, and considering both model parsimony and interpretability, we were informed by the Akaike Information Criterion (AIC), the Bayesian Information Criterion (BIC), the sample-size adjusted BIC (SSA-BIC), Entropy, the adjusted Lo-Mendell-Rubin test (LMR), and the bootstrapped parametric likelihood ratio tests (BLRT) to compare the model with K classes to a model with K-1 classes (Lo, Mendell, & Rubin, 2001; Asparouhov & Muthén, 2012; Morgan, 2015; UCLA, 2009).

Results

Participating Schools

From the pool of 689 schools invited to participate in the SSPESH, 598 (87%) Principals participated on behalf of their school. The survey took, on average, 10.7 minutes to complete based on the start and finish time-stamps. The representativeness of participating schools relative to all NSW schools was estimated using national school-level data on socio-demographic indices acquired from the Australian Curriculum, Assessment and Reporting Authority. Table 1 summarises the demographic characteristics of all NSW schools with a Year 6 enrolment in 2015 compared with SSPESH participating schools, firstly as distributions of unweighted data, and secondly as distributions after weighting by Year 6 enrolment and number of MCS participants per school. This shows that the 598 schools participating in the SSPESH were comparable on a range of socio-demographic indices to the total population of NSW schools with a Year 6 enrolment; all prevalences reported for the SSPESH participating schools (both unweighted and weighted estimates) lie within ~ 2% of NSW rates. The participating schools comprised two-thirds (66.9%) government and one-third (33.1%) non-government schools, mainly located in metropolitan (63.4%) and provincial (35.3%) communities. Schools were near-evenly distributed across socio-educational quartiles, with a small over-representation of socio-educational disadvantage (29.3% in lowest quartile). School communities were diverse, but on average comprised 23.7% of students with Language Background Other Than English (LBOTE) and 9.2% of students from Aboriginal or Torres Strait Islander background.
Table 1

Demographic characteristics of SSPESH participating schools relative to all NSW schools with a Year 6 student enrolment (unweighted, and weighted by enrolment)

Demographic measure

Unweighted averages

Weighted averagesa

 

NSW schools (n = 2371)

SSPESH schools (n = 598)

NSW schools (weighted)

SSPESH schools (weighted)

 

% (n)

% (n)

%

%

School sector

    

 Government

67.9 (1609)

66.9 (400)

67.4

66.4

 Non-government

32.1 (762)

33.1 (198)

32.6

33.6

Geographical location

    

 Metropolitan

59.9 (1421)

63.4 (379)

76.3

75.4

 Provincial

37.7 (894)

35.3 (211)

23.1

24.2

 Remote

1.8 (43)

1.3 (8)

0.4

0.4

 Very remote

0.5 (13)

0.0 (0)

0.1

0.0

 

Mean (SD)

Mean (SD)

Mean (SD)

Mean (SD)

ICSEA score

1007.7 (93.5)

1004.4 (88.7)

1033.2 (87.1)

1023.8 (84.1)

Socio-educational quartiles based on ICSEA (%)

    

 Lowest

28.8 (22.3)

29.3 (21.7)

23.5 (20.3)

25.2 (20.7)

 Lower-middle

24.3 (9.3)

24.9 (8.4)

22.9 (9.3)

23.9 (8.6)

 Higher-middle

23.4 (8.8)

23.6 (8.5)

24.7 (7.8)

24.7 (7.8)

 Highest

23.5 (21.7)

22.3 (20.4)

29.0 (23.4)

26.2 (21.5)

Proportion LBOTE (%)

23.3 (27.3)

23.7 (27.5)

31.1 (30.3)

30.0 (30.1)

Proportion indigenous (%)

9.1 (13.7)

9.2 (11.9)

6.0 (9.2)

6.5 (9.0)

Proportion female students (%)

48.6 (9.3)

48.9 (7.1)

48.5 (10.3)

48.8 (7.6)

NSW New South Wales; SSPESH Survey of School Promotion of Emotional and Social Health; ICSEA Index of Community Socio-Educational Advantage 2014 (a score derived from variables including parental school and non-school education and occupation, the school’s geographical location and proportion of Indigenous students; see ICSEA 2014: Technical Report. (http://www.acara.edu.au/_resources/ICSEA_2014_technical_report.pdf); LBOTE Language Background Other Than English

aTo estimate the proportions of children in NSW and SSPESH schools described by each demographic measure, weighting was applied based on the number of Year 6 students (NSW schools; 88,572 children) and Middle Childhood Survey participants in each school (SSPESH schools; 20,161 children); see Australian Curriculum, Assessment and Reporting Authority [2015]

Missing Data Imputation

Of the 598 participating schools, 556 (93.1%) had no missing data across the 14 statements. One school did not respond to the 14 statements and was excluded. The small amount (1%) of missing data from the 597 schools were well within acceptable ranges, and Little’s (1988) MCAR test suggested that missing values were completely at random across the 14 items (χ (274) 2  = 299.5, p > 0.10). Given limited missing values (in 41 schools), and a desire to retain as many schools as possible in the analysis, the missing data were imputed by replacing the missing value(s) with the rounded median score of the school’s responses to the non-missing items. This method preserved the categorical structure of the data and minimised possible bias relative to replacement with the series mean (Brick & Kalton, 1996). Statistical analyses of the 14 items were performed using the full database of n = 597 schools.

Structural Analyses

Item Distributions and Inter-Item Correlations

Across the 14 items, shown in Table 2, mean scores ranged from 1.27 to 2.54, within a possible range of 0–3, and the standard deviations ranged between 0.63 and 1.05. Most items exhibited non-normal distributions (skewness and kurtosis z-scores of over 3). All Spearman rank item correlation coefficients (see Supplementary Table 1) were significant (p < 0.01) and ranged from very weak (rs = 0.13) to strong (rs = 0.67). In accordance with our theoretical expectation, this suggested that the underlying factors would be moderately correlated, supporting our interest in deriving both a total scale and subscale scores across the four hypothesised conceptual domains.
Table 2

Descriptive statistics for the 14 SSPESH items (N = 597)

Item

Mean

SD

Skew

Kurtosis

1. Our school has specific policies and practices that promote inclusion and a safe environment

2.54

0.63

− 12.21

6.62

2. There is an effective leadership team in our school that has responsibility for student mental health and wellbeing (this may be one key person in very small schools)

2.48

0.72

− 13.39

7.05

3. Our policies, processes and procedures are reviewed annually to ensure they meet the needs of the school community

2.32

0.76

− 9.04

1.26

4. Students, parents, and staff in our school community regularly participate in providing structured feedback that informs school improvement

2.28

0.74

− 7.44

0.23

5. Student mental health and wellbeing is a regular item in our staff meetings

2.27

0.86

− 9.84

0.63

6. Social, emotional and resilience skills are taught at all year levels to all students in formally structured sessions

2.13

0.89

− 6.96

− 2.09

7. Our whole staff participate in opportunities to discuss child and adolescent development and the typical challenges they face

1.80

0.99

− 3.88

− 4.50

8. Activities that specifically engage parents, particularly those from diverse backgrounds, are regularly offereda

1.52

0.92

− 0.34

− 4.13

9. Activities for families that promote school-wide mental health and wellbeing are regularly offereda

1.27

0.94

1.75

− 4.50

10. We promote a wide range of information on student mental health to all parents and carers in a variety of ways (e.g. newsletters, website, forums)

1.59

0.94

− 0.14

− 4.56

11. Our school has clear referral pathways with local mental health services and supports families to access these services

2.33

0.81

− 10.76

2.66

12. Our school runs specific programs for students with additional needs

2.51

0.78

− 16.74

11.63

13. There is a budget allocation for supporting student mental health and wellbeing (e.g. for staffing, resources, parent information sessions)

1.91

1.05

− 5.22

− 4.87

14. Staff participate in training that develop their skills for communicating with students they are worried about, and their parents

1.73

0.97

− 2.87

− 4.43

aQuestionnaire item has been revised to provide examples. See additions in Supplementary Appendix

An initial Cronbach’s alpha reliability coefficient for the 14 items was α = 0.85. The item-total correlations for all but one item ranged between 0.4 and 0.6 (see Supplementary Table 1); the item-total correlation for item 4 was low (r = 0.31), and unlike the other items, deletion of Item 4 did not reduce Cronbach’s alpha, suggesting a 13-item scale (α = 0.85) for structural analysis. Item 4, ‘Students, parents, and staff in our school community regularly participate in providing structured feedback that informs school improvement’, was aimed at assessing schools’ data-driven decision making and, while important, conceptually stands alone from the other items.

Exploratory Factor Analyses

EFA was conducted initially on the 14-item scale, using data from the split-half subsample of 298 randomly selected schools, to examine the instrument’s dimensionality and to assess the need for removal of the item (Item 4) that was not contributing to the overall scale based on the item-total correlations. These analyses confirmed that Item 4 loaded independently of all other items. EFA was thus subsequently conducted on 13-items, fitting one to five factors using Oblimin rotation. The extraction of eigenvalues greater than 1.00 suggested a potential three-factor solution; however, this was not supported by the cross-loading of items onto multiple factors. On the basis of the fit statistics presented in Supplementary Table 2 (CFI, TLI, RMSEA, SRMR and χ2/df), the optimal solution was achieved with a 13-item scale loading onto four factors. This solution aligned with the theoretical model. The first factor explained 42.0% of the variance, with the four-factor model explaining 67.9% of the variance.

Supplementary Table 3 presents the four-factor EFA of the 13-item SSPESH scale, with the four domains conceptually interpreted as 1: Positive school community (PSC), 2: Student social emotional learning (SEL), 3: Engaging families (FAM), and 4: Supporting students experiencing difficulties (SUP). All items loaded above 0.4, with the exception of Items 2 and 5, where cross-loading was apparent; these items were assigned to the factor with the highest loading (for both items, this loading was 0.34). Correlations between the four factors ranged from r = 0.10 to 0.51, p > 0.05 (r1,2 = 0.26; r1,3 = 0.10; r1,4 = 0.21; r2,3 = 0.51; r2,4 = 0.48; r3,4 = 0.44). These results suggest that there was underlying unidimensionality, and the use of the instrument as a total scale was acceptable, alongside the derivation of total (summed) subscale scores reflecting each of the four factors. Acceptable internal consistency (alpha coefficients) for each factor was demonstrated, ranging from 0.67 to 0.72 (Gliem and Gliem 2003).

Confirmatory Factor Analysis

A four-factor CFA was specified on the 13 items using data from the other subsample of 299 randomly selected schools. This model fit the data well, with fit indices of CFI = 0.975, TLI = 0.987, RMSEA = 0.048, χ2/df = 1.675 (Hu & Bentler, 1999). Thus, the CFA yielded support for the four-domain solution and our conceptual model, as illustrated in Fig. 1.
Fig. 1

Four-factor model derived from confirmatory factor analysis (CFA) of the 13 SSPESH items, indicating standardised factor loadings for each item and inter-construct correlations (N = 299)

Criterion Validity

Independent t-tests conducted on the SSPESH total scale and the four subscale scores (sum of items loading on the factor) indicated significant differences between schools which had implemented KidsMatter Primary and those which had not, on the SSPESH total scale: t(595) = 3.38, p < 0.01; and the subscale scores of: Student social emotional learning: t(595) = 2.70, p = 0.01; Engaging families: t(595) = 3.57, p < 0.01; and Supporting students experiencing mental health difficulties: t(595) = 3.25, p < 0.01. There was no significant difference between schools on the Positive school community subscale: t(595) = 0.46, p = 0.65.

Latent Class Analysis

From the LCA testing of one through four class solutions (see Supplementary Table 4 for the fit statistics for each model), the preferred solution was a 3-class model with acceptable fit indices (AIC = 16,392.58; BIC = 16,915.22; SSA–BIC = 16,537.43; Entropy = 0.86); model fit was supported also by the adjusted LMR test (p = 0.019) and the bootstrapped parametric likelihood ratio test (p < 0.001). Average posterior probabilities were high (Class 1 = 0.93; Class 2 = 0.94, Class 3 = 0.94) and the average probability of being assigned to an incorrect class was low (0.07 or less). Based on the LCA of response patterns across the 13 items, 19.4% of schools were classified as Class 1, 53.7% of schools were Class 2, and 26.9% of schools were Class 3.

Figure 2 graphs the resulting three-class probability estimates for schools selecting each of the four response categories (i.e. the likelihood of scoring 0, 1, 2 or 3 against each item, depicted as lines). The mean scores, expressed as a percentage of the maximum possible score (3) for each item, are also shown (depicted as bars; described on the right vertical axes). The left vertical axes can be interpreted as the probability of selecting a particular response category and the horizontal axes present each of the 13 items.
Fig. 2

Three-class probability estimates, from latent class analysis, of being a high (class 1), moderate (class 2), or low (class 3) implementing school of emotional and social health promotion

Class 1 (19.4% of schools) showed a high probability of selecting Category 3 (Completely in place) and low probability of selecting Category 0 (Not yet in place) or Category 1 (Introducing) across most items, and was interpreted as high implementing schools. The first graph in Fig. 2 suggests that these schools were more likely to have Completely in place a positive school community, student social and emotional learning, and support for students experiencing difficulties, while engaging parents was still Taking hold.

The profile of Class 2 (53.7% of schools) showed a higher probability across many items of selecting Category 2 (Taking hold) and was interpreted as moderate implementing schools. The second graph in Fig. 2 suggests that these schools were more likely to have Completely in place a positive school community and some elements of support for students experiencing difficulties, while student social and emotional learning was Taking hold, and engaging families was being Introduced.

Class 3 (26.9% of schools) showed higher probability of selecting Category 0 or 1 across most items, and were interpreted as low implementing schools. The third graph in Fig. 2 suggests that in these schools, a positive school community was still Taking hold, while elements of supporting students experiencing difficulties were being Introduced or were Taking hold. Moreover, these schools were Introducing social and emotional learning in the curriculum, while engaging families was Not yet in place or was starting to be Introduced.

Identifying Scoring Thresholds

As a final step in the process, we applied the LCA grouping of schools to the summed scores to establish scoring thresholds (cut-points) for total scale and subscale scores. Using the response category scores (minimum = 0 to maximum = 3) for each of the 13 items, a total index score was calculated for each school to differentiate high and low performances (minimum possible score = 0, maximum = 39). The schools’ total scores ranged from a low score of 2 to a high score 39. The distribution of total scores within each class identified by the LCA is shown in Supplementary Figure 1. Inspection of the scores and class boundaries suggested thresholds set at the total scores of 21 for moderate implementing schools, and 33 for high implementing schools. Schools scoring less than 21 were interpreted as low implementing schools.

Scoring thresholds for SSPESH subscales were also calculated and are presented in Table 3 along with the thresholds for the total score. Threshold scores were adjusted to ensure that the sum of the subscale scores in each class fell within the total score ranges, in order to simplify interpretation for the end-user (i.e. researchers, schools). The variation across subscale scoring threshold indicated that the subscales of Positive school community and Student social emotional learning were relatively easier to implement, and thus had higher thresholds, compared to Engaging families and Supporting students experiencing mental health difficulties. Table 3 compares the resulting scoring bands and the distribution of Class 1 (high implementation), Class 2 (moderate implementation) and Class 3 (low implementation) schools in each scoring band. The SSPESH total score evidenced the most accurate classification, with 92% of schools on average, correctly allocated. The subscale Positive school community was less accurate, with 65% of schools, on average, being correctly allocated. The subscales of Student social emotional learning and Engaging families correctly allocated 68% of schools on average, and Supporting students experiencing difficulties correctly allocated 72% of schools on average.
Table 3

Scoring thresholds for the SSPESH total score and the four subscale scores at low, moderate or high implementation of mental health promotion, and their distribution across the high (class 1), moderate (class 2), and low (class 3) implementing classes derived from latent class analysis

 

Thresholds

Class 1 (%)

Class 2 (%)

Class 3 (%)

SSPESH total score (range 0–39)

 High

33–39

97

2

0

 Moderate

21–32

3

98

19

 Low

0–20

0

0

81

Positive school community subscale score (range 0–9)

 High

9

77

24

4

 Moderate

6–8

23

73

50

 Low

0–5

0

3

46

Student social emotional learning subscale score (range 0–9)

 High

9

54

8

0

 Moderate

5–8

46

82

34

 Low

0–4

0

10

66

Engaging families subscale score (range 0–9)

 High

7–9

69

12

0

 Moderate

3–6

31

77

41

 Low

0–2

0

11

59

Supporting students experiencing mental health difficulties subscale score (range 0–12)

 High

11–12

81

19

1

 Moderate

7–10

18

71

34

 Low

0–6

2

10

65

Bold values indicate correct allocation to class

Discussion

In this paper we describe the development of a new instrument, the Survey of School Promotion of Emotional and Social Health (SSPESH), which is designed to measure the school-level construct of whole-school emotional and social health promotion. The SSPESH offers the potential to profile schools and identify characteristics that differentiate states of high, moderate, and low implementation. The measure is also intended to serve as a guide for schools in identifying target areas for whole-school improvement in mental health and wellbeing in their community. Towards this end, an administrable version of the SSPESH, along with a scoring key to support interpretation, is provided in the Supplementary material.

Exploratory and confirmatory factor analysis supported the theoretical four-component structure of the 13-item version of the scale, with three items each related to Positive school community, Student social and emotional learning, and Engaging families, and four items related to Supporting students experiencing mental health difficulties. The preliminary psychometric properties obtained, provided support for the criterion-related validity of the scale based on its sensitivity to schools implementing KidsMatter Primary. Schools that were implementing this initiative were more likely to have in place policies and practices that promoted student wellbeing overall, and in three of the four components. However, there was no significant difference between schools implementing and not implementing KidsMatter on the Positive school community subscale, with most schools rating their capacity strongly within this domain. It is possible that the items comprising this domain may not have had sufficient discriminating power to avoid a ceiling effect.

The resulting 13-item SSPESH survey offers a brief instrument to assess a school’s current social and emotional health promotion activities, and to determine target areas for whole-school improvement, that will be easy for school leaders to use. A school’s overall social and emotional health promotion score is calculated by summing the scores on the 13 items, resulting in a theoretical range for the total scale score of 0–39. Latent class analysis identified three types of schools based on the characteristics of their response profiles to each of the 13 items. From these profiles, scoring thresholds were established to facilitate the interpretation of scores as an indication of a school’s current activities in whole-school mental (social and emotional) health promotion. The three ‘classes’ of school were described as ‘high implementing’ (19.4% of schools), ‘moderate implementing’ (53.7%), and ‘low implementing’ schools (26.9%). Scoring thresholds for the total score at 21 and 33 demarcated the moderate and high implementing schools, respectively.

Although less refined, a summed score was also calculated for each of the subscales to provide an indication of current capacity in the four SSPESH domains. The theoretical range for the three-item subscales of Positive school community, Student social and emotional learning and Engaging families is 0–9, and for the four-item subscale Supporting students experiencing difficulties is 0–12. Scoring thresholds vary across the subscales in response to the differential challenges schools have implementing each domain. These differences indicate that Principals submitted a range of ratings across the domains rather than appraise their school positively on every domain. Inspection of the probability profiles indicated that schools did not implement all four areas to the same extent, and showed that Engaging families was challenging, even for the high implementing schools. The domains of Positive school community and Student social and emotional learning are relatively easier for schools to implement and thus have higher scoring thresholds to distinguish moderate and high implementing schools (6 and 9, and 5 and 9, respectively). The lack of significant differentiation of Positive school community between schools that had and had not implemented KidsMatter Primary, and the likelihood of a ceiling effect, are reflected in the high scoring thresholds. The domains Engaging families and Supporting students experiencing difficulties have lower thresholds (3 and 7, and 7 and 11, respectively) to indicate moderate and high implementing schools. Accordingly, school leaders can use the SSPESH instrument to perform a brief assessment of their school, and using the threshold scores as a guide, have some understanding of their current activities in promoting emotional and social health, benchmarked against other schools.

Implications for School Mental Health Promotion

The development of a means to assess and quantify implementation of school promotion of emotional and social health offers the potential for fruitful new avenues of research and provides a user-friendly tool to help schools address the mental health needs of their community. The SSPESH can help briefly describe a school’s current functioning around emotional and social health promotion, aligned to contemporary health-promoting frameworks and initiatives in Australia and internationally. Such information could help identify where a school might benefit from a capacity-building intervention. A SSPESH assessment could help establish a school’s readiness to work with external services and health professionals on its path to establishing community partnerships. Moreover, schools could use the SSPESH survey as an evaluative guide to assess the comprehensiveness of whole-school programs in terms of the extent to which they address all of the domains. It might further be used over time to monitor a school’s progress on implementation of whole-school mental health promotion initiatives in relation to student wellbeing and academic outcomes, though further research is needed to determine the sensitivity of the instrument to measuring changes in implementation within schools. Finally, the present research has identified that, while schools find it relatively easy to build a positive school community, schools are most challenged by engaging and working effectively with parents. This signifies a shortcoming of current whole-school initiatives and calls for the development of specific resources and programs to better meet the needs of schools regarding parent engagement, as has been noted also by others (e.g. Fox & Olsen, 2014).

Limitations and Future Research

A limitation of this current study is that participating schools were only representative of one state in Australia and included only primary schools. Accordingly, we recommend caution in generalising findings to schools in other states and territories in Australia (and internationally) and to secondary schools. More representative data would allow generation of robust scoring thresholds and norms for the instrument and enable comparisons across schools in different settings (and could potentially be extended to include early childhood centres).

Another limitation was the approach used to establish criterion validity. While schools indicated that they were involved in implementing specific whole-school initiatives such as KidsMatter Primary, collection of information about the fidelity or duration of the initiatives’ implementation was beyond the scope of the study. Moreover, our sole reliance on the Principal’s report may have introduced response bias. Principals might have been inclined to over-estimate the extent to which their school was performing well. Principals might be encouraged to consult with other members of their school leadership team to form a collective view on the performance of the school. Nonetheless, schools that have engaged in a whole-school mental health program, such as KidsMatter Primary, are often more self-critical than other schools (Dix & Van Velsen, 2014). Such schools have a more comprehensive understanding of what is involved in achieving effective school-based mental health promotion (the Dunning-Kruger effect). Further study using the SSPESH alongside a more comprehensive community-level assessment, drawing on staff, parent, and student school satisfaction surveys, would further establish the validity of the instrument in primary and secondary schools. However, we acknowledge that the complexity of several items in the instrument, those which assess multiple components, may limit its utility to members of the school leadership only, with adapted versions to be developed for other informants. Likewise, the complexity of the item ‘Students, parents, and staff in our school community regularly participate in providing structured feedback that informs school improvement’ (Item 4 in the original, 14-item version administered) may have contributed to the lack of loading by this item on any of the four factors within the model.

While the current paper is limited to establishing the measurement properties of the SSPESH, there is opportunity to link this data with other school datasets. Data linkage would permit examination of the extent to which a school’s promotion of emotional and social health influences various outcomes, or is influenced by various factors. It offers the opportunity to address the question: Does whole-school mental health promotion make a difference to the social, emotional, behavioural and educational outcomes of children? Such work is envisaged to occur within the NSW-CDS cohort (Carr et al., 2016; Green et al., in press), via linkage to the self-reported Middle Childhood Survey mental health and wellbeing outcomes gathered from children at approximately 11 years of age, in their final year of primary schooling (Laurens et al., 2017), as well as multi-agency administrative records (Carr et al., 2016; Green et al., in press). Accordingly, examining the relationship between the SSPESH and student social emotional health over time is beyond the scope of the present paper, but will be explored in subsequent manuscripts from the study.

It is likely that schools in Australia and internationally will continue to be an important setting for addressing the mental health and wellbeing needs of children and adolescents. Schools that are highly engaged in whole-school emotional and social health promotion are likely to be more effective in providing for the psychosocial and educational needs of their whole student community. The SSPESH survey provides researchers and school leadership teams with an efficient and effective way to briefly assess and quantify the construct of whole-school promotion of emotional and social health.

Notes

Acknowledgements

The authors gratefully acknowledge support provided for the SSPESH administration by the NSW Department of Education (represented by Robert Stevens, Susan Harriman, and Liliana Ructtinger), the Catholic Education Commission NSW (represented by Peter Grace), the Association of Independent Schools of NSW (represented by Robyn Yates and Susan Wright), and other stakeholder organisations including the NSW Primary Principals’ Association, NSW Teachers Federation, Independent Education Union NSW/ACT, Federation of Parents and Citizens Associations of NSW, Council of Catholic School Parents, NSW Parents’ Council, Isolated Children’s Parents’ Association, and the NSW Aboriginal Education Consultative Group. The authors also acknowledge the contributions of all members of the NSW-CDS Scientific Committee who oversee the project: Vaughan Carr (Chairman), Miles Bore, Sally Brinkman, Marilyn Chilvers, Kimberlie Dean, Katherine Dix, Melissa Green, Felicity Harris, Allyson Holbrook, Maina Kariuki, Kristin Laurens, Rhoshel Lenroot, Kim Robinson, Maxwell Smith, Titia Sprague, Robert Stevens, Michael Tarren-Sweeney, Stacy Tzoumakis, and Megan Williams; as well as the support for SSPESH administration and/or data assembly provided by former members of the Committee: Stephanie Dick, Luke Duffy, Philip Hull, Luming Luo, Stephen Lynn, Brooke McIntyre, and Alessandra Raudino. The authors thank Principles Australia Institute for provision of the online survey. The information and perspectives expressed in the manuscript do not necessarily reflect the views of these contributing organisations or individuals.

Funding

This work was supported by funding from a National Health and Medical Research Council of Australia (NHMRC) Project Grant (1058652); an Australian Research Council (ARC) Linkage Project (LP110100150), with the NSW Ministry of Health, NSW Department of Education, and NSW Department of Family and Community Services representing the Linkage Project Partners; and an Australian Rotary Health Research Grant (RG104090). Harris, Carr, and Laurens were supported by funding from the Schizophrenia Research Institute, Australia, utilising infrastructure funding from the NSW Ministry of Health; Laurens was also supported by an ARC Future Fellowship (FT170100294); Green was supported by a NHMRC R. D. Wright Biomedical Career Development Fellowship (1061875); and, Dean by the Justice Health & Forensic Mental Health Network, NSW.

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.

Human and Animal Rights

All procedures performed in this study involving human participants were conducted in accordance with the ethical standards of the institutional and state research committee and with the 1964 Helsinki declaration and its later amendments. Informed consent was obtained from all individual participants included in the study. Information was gathered by school code, and Principals (or their representative) were not personally identified with their responses. Data was stored securely on password-protected servers, and access to the data was restricted to the research team. Ethical approval for the SSPESH was obtained from the UNSW Health Research Ethics Committee (Ref. No. HC 14348), in accordance with the National Statement on Ethical Conduct of Human Research, including associated Guidelines approved under Section 95A of the Privacy Act 1988 relevant to Catholic and Independent schools, and Section 41 of the Privacy and Personal Information Protection Act 1998 relevant to NSW public schools. Ethical approval was also obtained from the NSW Department of Education and Communities (SERAP 2015083).

Supplementary material

12310_2018_9280_MOESM1_ESM.docx (3.5 mb)
Supplementary material 1 (DOCX 3632 kb)

References

  1. Adelman, H. S., & Taylor, L. (2000). Promoting mental health in schools in the midst of school reform. Journal of School Health, 70(5), 171–178.CrossRefPubMedGoogle Scholar
  2. Ainscow, M., & West, M. (2006). Improving Urban Schools Leadership and Collaboration. Maidenhead: Open University Press.Google Scholar
  3. Asparouhov, T., & Muthén, B. (2012). Using Mplus TECH11 and TECH14 to test the number of latent classes. Mplus Web Notes, No. 14. https://www.statmodel.com/examples/webnotes/webnote14.pdf.
  4. Basch, C. E. (2011). Healthier students are better learners: High-quality, strategically planned, and effectively coordinated school health programs must be a fundamental mission of schools to help close the achievement gap. Journal of School Health, 81(10), 650–662.CrossRefPubMedGoogle Scholar
  5. Benningfield, M. M., Potter, M. P., & Bostic, J. Q. (2015). Educational impacts of the social and emotional brain. Child and Adolescent Psychiatric Clinics, 24(2), 261–275.CrossRefGoogle Scholar
  6. Beyondblue, (2017). An Evaluation of MindMatters—A Summary of Key Findings. Melbourne: Beyondblue.Google Scholar
  7. Brackett, M. A., Patti, J., Stern, R., Rivers, S. E., Elbertson, N. A., Chisholm, C., et al. (2009). A sustainable, skill-based approach to building emotionally literate schools. In M. Hughes, H. Thompson, & B. Bradford Terrell (Eds.), Handbook for Developing Emotional and Social Intelligence: Best Practices, Case Studies, and Strategies (pp. 329–358). New York: Pfeiffer.Google Scholar
  8. Brand, S., Felner, R. D., Shim, M., Seitsinger, A., & Dumas, T. (2003). Middle school improvement and reform: Development and validation of a school-level assessment of climate, cultural pluralism, and school safety. Journal of Educational Psychology, 95(3), 570–588.CrossRefGoogle Scholar
  9. Brick, J. M., & Kalton, G. (1996). Handling missing data in survey research. Statistical Methods in Medical Research, 5(3), 215–238.CrossRefPubMedGoogle Scholar
  10. Carr, V. J., Harris, F., Raudino, A., Luo, L., Kariuki, M., Liu, E., et al. (2016). New South Wales Child Development Study (NSW-CDS): An Australian multiagency, multigenerational, longitudinal record linkage study. BMJ Open.  https://doi.org/10.1136/bmjopen-2015-009023.Google Scholar
  11. CASEL. (2008). Social and emotional learning (SEL) and student benefits: Implications for the Safe Schools/Health Students core elements. In Collaborative for Academic, Social, and Emotional Learning. Chicago, IL: Author. https://files.eric.ed.gov/fulltext/ED505369.pdf.
  12. DCSF. (2009). Promoting and supporting positive behaviour in primary schools: Developing social and emotional aspects of learning (SEAL). In Department for Children, Schools and Families. Nottingham, UK: Author. http://dera.ioe.ac.uk/9437/.
  13. Dix, K. L. (2009). Identifying categories of student mental health with multiple measures and multiple informants using latent class analysis. In B. Matthews & J. A. Gibbons (Eds.), The Process of Research in Education: A Festschrift in Honour of John P Keeves AM (pp. 135–150). Adelaide: Shannon Research Press.Google Scholar
  14. Dix, K. L., Keeves, J. P., Slee, P. T., Lawson, M. J., Russell, A., Askell-Williams, H., et al. (2010). The KidsMatter Primary Evaluation: Technical Report and User Guide. Adelaide: Shannon Research Press.Google Scholar
  15. Dix, K. L., Slee, P. T., Lawson, M. J., & Keeves, J. P. (2012). Implementation quality of whole-school mental health promotion and students’ academic performance. Child and Adolescent Mental Health, 17(1), 45–51.  https://doi.org/10.1111/j.1475-3588.2011.00608.x.CrossRefPubMedPubMedCentralGoogle Scholar
  16. Dix, K. L., & Van Velsen, J. (2014). Best Practice Blended Learning: The KidsMatter Portal Evaluation. Adelaide: Principals Australia Institute.Google Scholar
  17. Domitrovich, C. E., Bradshaw, C. P., Poduska, J. M., Hoagwood, K., Buckley, J. A., Olin, S., et al. (2008). Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework. Advances in School Mental Health Promotion, 1(3), 6–28.  https://doi.org/10.1080/1754730X.2008.9715730.CrossRefPubMedPubMedCentralGoogle Scholar
  18. Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psycholology, 41(3–4), 327–350.  https://doi.org/10.1007/s10464-008-9165-0.CrossRefGoogle Scholar
  19. Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82(1), 405–432.CrossRefPubMedGoogle Scholar
  20. E4L. (2017). The Toolkit: Social and Emotional Learning. http://evidenceforlearning.org.au/toolkit/social-and-emotional-learning/.
  21. ENSEC. (2009). European Network for Social and Emotional Competence. http://www.enseceurope.org/.
  22. Fan, X., Thompson, B., & Wang, L. (1999). Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 56–83.  https://doi.org/10.1080/10705519909540119.CrossRefGoogle Scholar
  23. Feigenberg, L. F., Watts, C. L., & Buckner, J. C. (2010). The school mental health capacity instrument: Development of an assessment and consultation tool. School Mental Health, 2(3), 142–154.  https://doi.org/10.1007/s12310-010-9041-6.CrossRefGoogle Scholar
  24. Flesch, R. (1980). The ABC of Style: A Guide to Plain English. New York: HarperCollins.Google Scholar
  25. Fox, S., & Olsen, A. (2014). Education Capital: Defining Parental Engagement. Melbourne: ACT: Australian Research Alliance for Children and Youth.Google Scholar
  26. Fullan, M. (2007). New Meaning of Educational Change (4th ed.). New York: Columbia University.Google Scholar
  27. Gillard Offers Cash to Improving Schools. (2010). Australian Broadcasting Corporation.Google Scholar
  28. Gliem, J. A., & Gliem, R. R. (2003). Calculating, interpreting, and reporting Cronbach’s alpha reliability coefficient for Likert-type scales. Paper Presented at the Midwest Research-to-Practice Conference in Adult, Continuing, and Community Education, United States. Google Scholar
  29. Graetz, B., Littlefield, L., Trinder, M., Dobia, B., Souter, M., Champion, C., et al. (2008). KidsMatter: A population health model to support student mental health and well-being in primary schools. International Journal of Mental Health Promotion, 10(4), 13–20.  https://doi.org/10.1080/14623730.2008.9721772.CrossRefGoogle Scholar
  30. Green, M. J, Harris, F., Laurens, K. R., Kariuki, M., Tzoumakis, S., Dean, K., et al. (in press). The New South Wales child development study (NSW-CDS)—Wave 2 (Child age 13 years). International Journal of Epidemiology.Google Scholar
  31. Green, J. G., McLaughlin, K. A., Alegría, M., Costello, E. J., Gruber, M. J., Hoagwood, K., et al. (2013). School mental health resources and adolescent mental health service use. Journal of the American Academy of Child and Adolescent Psychiatry, 52(5), 501–510.  https://doi.org/10.1016/j.jaac.2013.03.002.CrossRefPubMedPubMedCentralGoogle Scholar
  32. Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modelling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6(1), 53–60.Google Scholar
  33. Horner, R. H., Todd, A. W., Lewis-Palmer, T., Irvin, L. K., Sugai, G., & Boland, J. B. (2004). The school-wide evaluation tool (SET): A research instrument for assessing school-wide positive behaviour support. Journal of Positive Behavior Interventions, 6(1), 3–12.CrossRefGoogle Scholar
  34. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.  https://doi.org/10.1080/10705519909540118.CrossRefGoogle Scholar
  35. IBM Corp. (2013). IBM SPSS Statistics for Windows, Version 22.0. Armonk, NY: IBM Corp.Google Scholar
  36. Jensen, B., & Sonnemann, J. (2014). Turning Around Schools: It Can Be Done. Melbourne: Grattan Institute.Google Scholar
  37. Kidger, J., Araya, R., Donovan, J., & Gunnell, D. (2012). The effect of the school environment on the emotional health of adolescents: A systematic review. Pediatrics, 129(5), 925–949.  https://doi.org/10.1542/peds.2011-2248.CrossRefPubMedGoogle Scholar
  38. KidsMatter. (2009). KidsMatter Primary Framework. https://www.kidsmatter.edu.au/primary/about-kidsmatter-primary/framework.
  39. KidsMatter. (2013). How to Guide for Schools on Surveys and the Planning Process. Commonwealth of Australia. https://www.kidsmatter.edu.au/primary/surveys.
  40. KidsMatter (2017). School Surveys. http://www.kidsmatter.edu.au/primary/surveys.
  41. Langford, R., Bonell, C. P., Jones, H. E., Pouliou, T., Murphy, S. M., Waters, E., et al. (2014). The WHO Health Promoting School framework for improving the health and well-being of students and their academic achievement (Review). Cochrane Database of Systematic Reviews, 4(4), CD008958.  https://doi.org/10.1002/14651858.CD008958.pub2.Google Scholar
  42. Laurens, K. R., Tzoumakis, S., Dean, K., Brinkman, S. A., Bore, M., Lenroot, R. K., et al. (2017). The 2015 middle childhood survey (MCS) of mental health and well-being at age 11 years in an Australian population cohort. British Medical Journal Open, 7(6), e016244.Google Scholar
  43. Leighton, J. P. (2017). Using Think-Aloud Interviews and Cognitive Labs in Educational Research. Oxford: Oxford University Press.CrossRefGoogle Scholar
  44. Little, R. J. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83, 1198–1202.CrossRefGoogle Scholar
  45. Lo, Y., Mendell, N., & Rubin, D. (2001). Testing the number of components in a normal mixture. Biometrika, 88, 767–778.CrossRefGoogle Scholar
  46. Mellin, E. A., Taylor, L., Weist, M. D., & Lockhart, N. C. (2016). The expanded school mental health collaboration instrument [Community Version]: Development and initial psychometrics. School Mental Health, 8(2), 305–318.  https://doi.org/10.1007/s12310-015-9164-x.CrossRefGoogle Scholar
  47. Morgan, G. B. (2015). Mixed mode latent class analysis: An examination of fit index performance for classification. Structural Equation Modeling: A Multidisciplinary Journal, 22(1), 76–86.  https://doi.org/10.1080/10705511.2014.935751.CrossRefGoogle Scholar
  48. Muthén, L. K., & Muthén, B. O. (2007). Mplus User’s Guide (5th ed.). Los Angeles, CA: Muthén and Muthén.Google Scholar
  49. Novins, D. K., Green, A. E., Legha, R. K., & Aarons, G. A. (2013). Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review. Journal of the American Academy of Child and Adolescent Psychiatry, 52(10), 1009–1025.CrossRefPubMedPubMedCentralGoogle Scholar
  50. Payton, J., Weissberg, R. P., Durlak, J. A., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B., et al. (2008). The Positive Impact of Social and Emotional Learning for Kindergarten to Eighth-Grade Students: Findings from Three Scientific Reviews. Technical Report. Chicago, IL: Collaborative for Academic, Social, and Emotional Learning (NJ1).Google Scholar
  51. Resnick, L. (2010). Nested learning systems for the thinking curriculum. Educational Researcher, 39, 183–197.CrossRefGoogle Scholar
  52. Sammons, P. (2007). School Effectiveness and Equity: Making Connections. Reading: CfBT.Google Scholar
  53. Simonsen, B., Eber, L., Black, A. C., Sugai, G., Lewandowski, H., Sims, B., et al. (2012). Illinois statewide positive behavioural interventions and supports: Evolution and impact on student outcomes across years. Journal of Positive Behavior Interventions, 14, 5–16.CrossRefGoogle Scholar
  54. Sklad, M., Diekstra, R., Ritter, M. D., Ben, J., & Gravesteijn, C. (2012). Effectiveness of school-based universal social, emotional, and behavioral programs: Do they enhance students’ development in the area of skill, behavior, and adjustment? Psychology in the Schools, 49(9), 892–909.CrossRefGoogle Scholar
  55. Slee, P. T., Lawson, M. J., Russell, A., Askell-Williams, H., Dix, K. L., Owens, L., et al. (2009). The KidsMatter Evaluation: Final Report. Melbourne: Beyondblue.Google Scholar
  56. Slee, P. T., Murray-Harvey, R., Dix, K. L., Skrzypiec, G., Askell-Williams, H., Lawson, M. J., et al. (2012). KidsMatter Early Childhood Evaluation Report. Adelaide, SA: Shannon Research Press.Google Scholar
  57. Staten, L. K., Teufel-Shone, N. I., Steinfelt, V. E., Ortega, N., Halverson, K., & Flores, C., et al. (2005). The School Health Index as an Impetus for Change. http://www.cdc.gov/pcd/issues/2005/jan/04_0076.
  58. Taylor, R. D., Oberle, E., Durlak, J. A., & Weissberg, R. P. (2017). Promoting positive youth development through school-based social and emotional learning interventions: A meta-analysis of follow-up effects. Child Development, 88(4), 1156–1171.CrossRefPubMedGoogle Scholar
  59. Trinder, M., Roberts, B., & Cavanagh, S. (2009). Impressive evaluation results lead to significant expansion of KidsMatter. Psych: The Bulletin of the Australian Psychological Society, 31(6), 22–24.Google Scholar
  60. Tseng, V., & Seidman, E. (2007). A systems framework for understanding social settings. Am Journal of Community Psychology, 39(3–4), 217–228.  https://doi.org/10.1007/s10464-007-9101-8.CrossRefGoogle Scholar
  61. UCLA. (2009). Latent Class Analysis: Mplus Data Analysis Examples. University of California Los Angeles Institute for Digital Research and Education. https://stats.idre.ucla.edu/mplus/dae/latent-class-analysis/.
  62. Weissberg, R. P., Goren, P., Domitrovich, C., & Dusenbury, L. (2013). CASEL Guide Effective Social and Emotional Learning Programs: Preschool and Elementary School Edition. Chicago, IL: CASEL.Google Scholar
  63. Weist, M. D., Stephan, S., Lever, N., Moore, E., & Lewis, K. (2006). School Mental Health Quality Assessment Questionnaire (SMHQAQ). Baltimore: Center for School Mental Health Analysis & Action.Google Scholar
  64. Wells, J., Barlow, J., & Stewart-Brown, S. (2003). A systematic review of universal approaches to mental health promotion in schools. Health Education, 103(4), 197–220.  https://doi.org/10.1108/09654280310485546.CrossRefGoogle Scholar
  65. WHO. (2004). Promoting Mental Health: Concepts, Emerging Evidence, Practice: Summary Report. Geneva: World Health Organization.Google Scholar
  66. Wyn, J., Cahill, H., Holdsworth, R., Rowling, L., & Carson, S. (2000). MindMatters, a whole-school approach promoting mental health and wellbeing. Australian and New Zealand Journal of Psychiatry, 34, 594–601.CrossRefPubMedGoogle Scholar
  67. Zins, J. E., Bloodworth, M. R., Weissberg, R. P., & Walberg, H. J. (2004). The scientific base linking social and emotional learning to school success. In J. E. Zins, R. P. Weissberg, M. C. Wang, & H. J. Walberg (Eds.), Building Academic Success on Social and Emotional Learning: What Does the Research Say? (pp. 3–22). New York, NY: Teachers College Press.Google Scholar

Copyright information

© The Author(s) 2018

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.Australian Council for Educational ResearchAdelaideAustralia
  2. 2.School of PsychiatryUniversity of New South WalesRandwickAustralia
  3. 3.Neuroscience Research AustraliaRandwickAustralia
  4. 4.School of Social SciencesUniversity of New South WalesSydneyAustralia
  5. 5.Justice Health & Forensic Mental Health NetworkMalabarAustralia
  6. 6.Department of Psychiatry, School of Clinical SciencesMonash UniversityClaytonAustralia
  7. 7.School of PsychologyAustralian Catholic University (Brisbane)VirginiaAustralia

Personalised recommendations