Abstract
Educational researchers have increasingly recognised the importance of school climate as a malleable factor for improving academic performance. In this perspective, we exploit the data collected by the Italian Institute for the Evaluation of the Education System (INVALSI) to assess the effect of some school climate related factors on academic performance of tenth-grade Italian students. A Multilevel Bayesian Structural Equation Model (MBSEM) is adopted to highlight the effect of some relevant dimensions of school climate (students’ disciplinary behaviour and parents’ involvement) on academic performance and their role on the relationships between student socioeconomic status and achievement. The main findings show that disciplinary behaviour, on the one hand, directly influences the level of competence of the students, and, on the other hand, it partly mediates the effect of socioeconomic background whereas parents’ involvement does not appear to exert any significant effect on students’ performance.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
In recent years, there has been a remarkable consensus among educators, policymakers and scholars in the identification of school climate as one of the main levers that can be used to improve national school systems. The first studies describing the positive effects of school climate on the learning process date back to the early 1900s (Perry 1908). Later, Halpin and Croft (1963) started the first empirical analysis of school climate, prompting a rich and broad line of research systemically studying the influence of school climate on student learning and development. Nowadays, most of the existing papers regarding this topic, however, refer to the school system of the United States, where the debate on school climate began over 50 years ago. For example, Sherblom et al. (2006) studied the relationship between aspects of school climate, as reported by students, teachers/staff and parents, and standardised tests in reading and mathematics for third and fourth grade students. Brand et al. (2003) analysed a large-scale sample of over 105,000 students in 188 schools and suggested that school climate dimensions were consistently associated with academic, behavioural, and socioemotional adjustment indices. The results by Stewart (2008) suggest that school climate—in particular, the sense of school cohesion felt by students, teachers, and administrators—is a strong predictor of successful student outcomes. Similarly, the analysis by MacNeil et al. (2009) shows that students achieve higher scores on standardised tests in schools with healthy learning environments. Generally, the learning process can be sustained by a positive and respectful atmosphere (Harris and Chrispeels 2006; Kyriakides and Creemers 2008). Moreover, some longitudinal studies (Hoy et al. 1998) have confirmed that a positive school climate not only contributes to immediate student achievement but also to effects that seem to persist for years.
Another line of research includes several analyses based on large international student assessment data from the OECD Programme for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS) carried out by the International Association for the Evaluation of Educational Achievement (IEA). These data provide opportunities to make comparisons between countries using standardised and high quality information. Additionally, they allow the analysis to be extended not only to school climate but also to factors that characterise the learning processes at the student and school level. Note, however, that exploring specific aspects, such as school climate, requires ad hoc surveys and methodological tools that can hardly be carried out on a large scale. Consequently, most studies based on international databases focus on a few specific dimensions of school climate, such as those related to disciplinary aspects, which are the characteristics most investigated by these surveys.
Despite the strong consensus regarding the importance of a positive climate on educational outcomes, some caution must be adopted in analysing and interpreting its effect. It should be emphasised that school climate can be endogenously linked to other factors, in particular those of a socioeconomic nature, which, in turn, affect school outcomes directly. For example, although there are clear signals of a strong and meaningful relationship between school climate and cognitive outcomes, the interpretation of these relationships should account for the school socioeconomic composition, which is simultaneously related to school climate and academic achievement, as shown by several studies (Ma and Willms 2004; OECD 2010; Rangvid 2007).
Consequently, the analysis of school climate effects must use appropriate statistical methods that can account for the complex structures of relationships that characterise educational processes. From this perspective, our primary research objective is to determine whether some school climate-related aspects can influence the academic outcomes of Italian students by accounting for three main dimensions that characterises the concept of school climate using the data available on Italian students. More precisely, we adopt a Multilevel Bayesian Structural Equation Model (MBSEM) approach to measure three latent multidimensional constructs of school climate through several observed indicators and to assess its influence on academic achievement. The main objective is to assess the causal effect of school climate on students’ outcomes to fill a gap in empirical analyses which, with a few exceptions (e.g., Benbenishty et al. 2016), highlight only correlational links. To identify the causal effect from non-experimental cross-sectional data, we include the Socio-Economic Status (SES) of the students as a measured common cause influencing both school climate and academic performance. Conditioning on SES helps in reducing the source of spurious correlation between school climate and performance thus addressing the endogeneity issue in the relationship between climate and performance and recovering the correct causal estimate (Antonakis et al. 2010; Pearl 2012). Using data from the Italian Institute for the Evaluation of the Education System (INVALSIFootnote 1), this paper contributes to the debate on school climate policies and provides evidence on the role of some aspects related to school climate in the Italian educational context, wherein there is scant literature on this issue.
The paper tries to verify whether there is space to extend the role of school managers to implement effective practices aimed at improving student performance by acting on school climate. In this light, the analysis of school climate is relevant, as school climate can be fully counted among malleable factors, i.e., factors that schools can easily manipulate (Voight et al. 2013). Acting on this factor may result in effective policies that do not need broader system reforms, which can be particularly relevant in school systems, such as the Italian school system, where constraints of autonomy and budgets limit the number of policy levers for academic improvement.
In detail, we aim to analyse (i) the influence of relevant dimensions of school climate on academic performance, namely parents’ involvement in interactions with teachers (PI), parents’ participation in school activities and organisation (PP) and students’ disciplinary behaviour (SDB); (ii) the role of these factors as mediators of the effect of socioeconomic background on performance.
Specifically, our research hypotheses are the following:
H1: The three investigated dimensions of school climate positively and significantly affect student achievement.
H2: The socio economic status influences these aspects of school climate, which in turn influence academic achievement, thus pointing to an indirect effect (mediation).
A positive and significant indirect effect of socio-economic background through a specific aspect of school climate would imply that one can use this aspect as a tool to improve academic performance.
This paper is structured in the following way. The next section explores the multidimensional concept of school climate and it describes in detail the dimensions that can be analysed in the Italian context on the basis of the available data. The methodology and data are described in Sect. 3. Section 4 reports the empirical results while Sect. 5 presents the conclusions.
2 Relevant dimensions of the school climate: a focus on students’ disciplinary behaviour and parental involvement
Despite the broad consensus on the positive effect of the school climate both on academic achievement and healthy behaviour for students (Patton et al. 2006), there is not a universally agreed-upon definition of school climate. The multidimensionality of this concept has led educators and researchers to propose, and use, a wide range of school climate definitions, each focused on specific aspects of this complex construct. Consequently, definitions and measures of school climate have evolved over the past decades. Halpin and Croft (1963) defined school climate in terms of teachers’ perceptions of the school’s personality, i.e., the teachers’ views of school routines, behaviours, and interactions among teachers and administrators. Since this seminal work, the concept of school climate was expanded to include the perceptions of students and other school stakeholders, such as parents and community members (Pyper et al. 1987; Simons-Morton and Crump 2003; Bear et al. 2014; Brand et al. 2008).
Especially in the US, the growing interest in school reforms focused on improving the school climate has led to the development of ad hoc surveys and tools to provide proxy measures of school climate. Since 2004, for example, the California School Climate Survey (CSCS) has been administered to assess school staff perceptions about learning and teaching conditions on many aspects of school climate, such as resource provisions and training, school safety, professional development, student learning environment and student expectations.
Some measures of school climate have been proposed by organizations or groups of researchers; for example, the Comprehensive School Climate Inventory (CSCI) is a scientifically sound and helpful tool suggested by the National School Climate Center (Clifford et al. 2012; Guo et al. 2011) which provides feedback on how students, staff, and parents perceive their school’s climate for learning. The National School Climate Center defines the school climate as the quality of school life considering the norms and values, interpersonal relations and social interactions, and organizational process and structures.
Another example of a descriptive measure is the Organizational Climate Index (OCI). This index, proposed by Hoy et al. (2002), analyses four dimensions of school climate: principal leadership, teacher professionalism, achievement press for students to perform academically, and vulnerability to the community. Additionally, Sink and Spencer (2007) suggested the Teacher Version of My Class Inventory, which assesses teachers’ perceptions of the classroom climate across five dimensions: (a) overall student satisfaction with the learning experience, (b) peer relations, (c) difficulty level of classroom materials, (d) student competitiveness, and (e) school counsellor impact on the learning environment.
In conclusion, the concept of school climate has undergone numerous changes over the last few decades, resulting in various definitions affecting how the construct is measured and analysed. Thus, a critical first step for any analysis of school climate starts with defining a construct so that reliable and valid measurement and evaluation can occur (Cohen 2006). In this light, another critical aspect is the availability of surveys, and consequently of data, focused on school climate. Indeed, apart from the U.S. experience, in European and extra-European countries there is a limited availability of data, which does not allow a full description and interpretation of the effect of school climate with consideration of all its many facets. This problem is particularly relevant, especially in Italy, where the school assessment system is relatively young (Castellano and Longobardi, 2020) and many dimensions of educational processes, such as school climate, are not yet adequately investigated.
In this perspective, the availability of data related to the Italian context of this specific topic does not allow to measure and analyse the different dimensions that can be traced back to the multifaceted concept of school climate but only to focus on the role that some relevant aspects of this multidimensional construct play in determining student performance. Specifically, we consider three peculiar dimensions of school climate. The first dimension (PI) concerns the degree of interaction between parents and teachers (e.g. how often parents attend general meeting or parent-teacher conferences), while the second dimension (PP) regards the direct participation of parents in school activities and organization (e.g. whether parents discuss about school funds and/or school structures, contribute to define school programmes or participate in educational activities). The third dimension includes aspects linked to students’ disciplinary behaviour (SDB) such as truancy (being late for school and making unexcused absences) and bullying or intimidating behaviour either among students or with respect to teachers.
Although these aspects do not exhaust the wide range of dimensions underlying the concept of school climate, they are certainly relevant because a large literature has analysed their role for improving the student performance.
The strand of literature that investigated the role of parental involvement in determining a positive school climate, and therefore improving the student achievement, is quite broad.
Parental involvement can be interpreted either as home based or school based involvement (Borgonovi and Montt, 2012). The former includes the interactions between parents and their child and it regards helping their child with homework as well as discussing education matters or caring about health and safety of children, etc. The latter refers to the interactions between parents and the school staff by focusing on how often the parents interface with the teachers to follow the school progress of their children, whether they participate in the school governance or in extracurricular/voluntary activities in the school.
In recent years, policy makers have placed great importance on the role of parental involvement as a lever to improve students’ school performance and numerous efforts have been taken in many countries to integrate the parent involvement into broader educational initiatives. In particular, in the US experience, parental involvement has been one of the cornerstones of major educational reforms such as President Obama’s “Race to the Top” or President Bush’s “No Child Left Behind”.
Empirical evidence of the relationships between parent involvement and academic achievement is equivocal. Some findings support the claim of a positive correlation between the active participation of parents and their children’s academic achievement (Jeynes, 2012; Zellman and Waterman 1998; Burcu and Sungur 2009), other studies highlight the positive effect of parental involvement on social skills (Sheridan et al., 2012), good behaviour (Sheridan et al., 2017) and mental health (Wang and Sheikh-Khalil, 2014). Conversely, many scholars indicates that parent involvement is only modestly associated with student outcomes or has no effect on achievement (e.g. Domina 2005; El Nokali et al., 2010; Epstein 1991; Fan 2001).
Different reasons can explain this apparent inconsistency, for example “…a high level of parental involvement in some school activities, such as volunteering in physical and extracurricular activities, may reflect a lack of school resources” (OECD, 2019) or a high degree of interaction between teachers and parents may occur on the initiative of teachers due to a poor school performance of the child.
As to the student disciplinary behaviour, a series of correlational studies, based on PISA data, have shown that the disciplinary behaviour is positively associated with academic achievement. For example, Shin et al. (2009) suggest that in Japan, Korea and the United States there is a strong correlation between disciplinary climate and mathematics performance. Güzel and Berberoğlu (2005) show that the influence of students’ perceptions of classroom disciplinary climate on their reading achievement is positive in Japan, non-significant in Norway, and negative in Brazil. Ma et al. (2013) show that in some Asian countries, disciplinary climate has a positive association with student performance in reading, mathematics, and science literacy. Similarly, Ning et al. (2015) report that the classroom disciplinary climate of schools can explain 11% of the between-school differences in reading achievement over countries.
3 Data and methods
3.1 Data and procedure
In Italy, education is compulsory from 6 to 16 years of age, and it is divided into three main cycles: (i) pre-primary school for children between 3 and 6 years of age; (ii) the first cycle of education, including primary education (5 years) and lower secondary school (3 years); (iii) the second cycle of education (5 years). The second cycle of education includes three types of upper-secondary school: schools with an academic focus that mainly cover humanities and scientific fields (licei), technical schools and vocational schools.
Each year, all the students of different school gradesFootnote 2 are subjected to external evaluation by the INVALSI through standardised tests in two domains: reading and mathematics. The main aim of these tests is to help the school principal and the teachers evaluate the performance of their classes and students. Additionally, the INVALSI results help the central government understand the general performance of the school system and make intra- and inter-school comparisons in space (at the national, regional, and provincial level) and time. INVALSI not only evaluates the students’ skills but also requires pupils to answer a questionnaire (Student questionnaire) about their family background, which increases the information potential of the data. Although the INVALSI tests are exhaustively administered, for each school level a random sample is extracted with a two-stage method: in the first stage the schools are sampled and in the second stage two whole classes for each school selected at the previous stage are sampled. The purpose of this sampling is to guarantee the reliability of the data collected: in the sample classes there is in fact an external observer with the task of guaranteeing the regularity of the administration of the tests and avoiding that the data are biased by the presence of teacher and/or student cheating (Longobardi et al. 2018). On this sample of schools, INVALSI administers a School questionnaire, completed by the school principal, to collect important information related to various aspects of school life, such as management practices, availability of infrastructure and resources and school climate.
As mentioned in previous sections, the aim of our analysis is to explore the relationships between academic performance (in terms of the students’ score in reading and mathematics test, our target variables) and the three dimensions of school climate while accounting for a set of exogenous characteristics; in this light our dataset is the result of a merging that involves (i) data from INVALSI school questionnaire filled by school principals, (ii) the results from INVALSI tests assessing mathematical and reading skills, and (iii) data related to socio economic variables collected by the INVALSI student questionnaire.
Specifically, the analysis was conducted considering the data related to a nationally representative sample of 642 upper secondary schools (tenth grade) with 23,673 students for which the variables obtained from the school principals’ questionnaire are available.
A general multilevel SEM approach (MSEM, Muthén and Asparouhov 2008; Preacher et al. 2010) is adopted to account for the data hierarchical structure, students (Level-1 units) nested in schools (Level-2 units). This methodology allows to account for both sources of variability (within-school and between-school in our case) and separate between-school and within-school effects, without introducing bias.
The general MSEM model splits each observed Level-1 variable into two latent (within and between) components (Preacher et al. 2010). In our case, every student-level variable is treated as jointly caused by within- and between-school variation: this means that the observations referring to different students in the same school have to be considered as multiple indicators of the latent school-level construct. In addition, at every level multiple indicators can be used to reflect an underlying unobserved construct as in the measurement model of a common SEM approach. In these terms, the general MSEM framework appear as a doubly latent model (Marsh et al. 2012).
3.2 Measures
Consistently with the analysed literature, and on the basis of the items of the school questionnaire, we consider three aspects of school climate representing three distinct latent dimensions. The first two refer to the parental involvement: the degree of interaction between parents and teachers (PI); the participation of parents in school activities and organization (PP) while the third dimension is the students’ disciplinary behaviour (SDB)Footnote 3. The PI dimension (interactions with teachers) is measured by the school principal’s answers to the following items: (i) participation in parent meetings; (ii) voting for the election of school council; (iii) participation in parent-teacher conferences; (iv) meeting the teachers about the progress and behaviours of children. The PP dimension (participation in school activities and organization) is measured by the school principal’s answers to the following items: (i) volunteering in building maintenance activities or other outdoor spaces activities; (ii) discussing the academic achievement of all the students; (iii) discussing school fund spending; (iv) discussing the conditions of school structures and buildings; (v) contributing money to good school performance; (vi) volunteering for good school performance; (vii) contributing to the definition of school programmes; (viii) participating actively in educational activities; (ix) discussing the results of the INVALSI assessment tests. These 13 questions were answered on a 4-point Likert-type response option scale (1 = not at all; 2 = little; 3 = to some extent; 4 = a lot). Higher values suggest higher participation of parents in the various aspects of their children’s school life.
For student disciplinary behaviour, the responses of the school principal concern the following items: (i) student lateness; (ii) student truancy; (iii) disruptive behaviours in the classroom; (iv) students bullying other students; (v) students intimidating or offending other students; (vi) aggressive behaviours towards teachers or school staff; (vii) students have discontinuous frequency; (viii) students commit vandalism. These items are also measured on a 4-point Likert-type response option scale (1 = not at all; 2 = little; 3 = to some extent; 4 = a lot), where higher values denote a greater frequency of negative student behaviours.
The frequency distributions of the indicators of the three latent constructs are displayed in Table 1. According to the school principal, parents are most involved in meeting the teachers and least involved in contributing to the definition of school programmes and in school building maintenance activities. The most recurrent negative students’ behaviours are vandalism and bullying, whereas arriving late is deemed to be the least frequent.
Table 2 includes the descriptive statistics of the target variable (test scores in reading and maths), the student socio-economic status (SESFootnote 4) and a set of exogenous covariates that typically characterise the Education Production Function (EPF) models (Niederle and Vesterlund 2010; Buchmann and Parrado 2006), namely, gender, grade retention (regular or not) and immigrant status. In addition, four geographical areasFootnote 5 are considered to account for the territorial literacy divide (Quintano et al. 2012), i.e. the wide territorial difference in achievement between Northern and Southern students; this gap is confirmed by several studies which show that students at schools in Northern Italy tend to obtain higher achievement scores than their counterparts in other regions, all else being equal (Giambona and Porcu 2018).
3.3 Data analysis plan
The first step of our analysis is to test whether each dimension of school climate (PI, PP, SDBFootnote 6) is accurately measured by the corresponding indicators. In this light, we perform a correlated traits confirmatory factor analysis (CFA) with three latent continuous factors, measured by 21 dependent items. For the correlated traits CFA model, each item is specified to load onto only the factor it is designed to measure. The correlations among the three factors are freely estimated.
The second step consists in the estimation of the multilevel SEM with the aim of addressing our research hypotheses (Fig. 1). Socio-Economic Status (SES) and test scores (Maths Achievement, MA, and Reading Achievement, RA) are student-level (or level-1) variables. It is assumed that they have within- and between-school components. Therefore, they are split into latent within and between components. This means that, in addition to the effect of SES on individual scores within schools, at the school level (or level-2) the mean SES has an effect on the mean scores. Both scores have been included in the model as endogenous variables, and they are allowed to be correlated. At the school level, the effect of SES on the scores is mediated by Parent Involvement (PI), Parent Participation (PP) and Student Disciplinary Behaviour (SDB) that are school-level latent variables with multiple categorical indicators. This is a mediation model with a 1-(2-2-2)-1 design according to Preacher et al. (2010).
To model Level-2 categorical variables, we use a Bayesian estimator with non-informative priors and a Markov chain Monte Carlo (MCMC) algorithm based on the Gibbs sampler. In the case of the estimation of complex multilevel SEMs with categorical indicators, the Bayes estimator provides a valid full-information alternative to the classical weighted least squares mean and variance adjusted (WLSMV) estimation procedures (Asparouhov & Muthén 2010; Muthén & Asparouhov 2012).
Every structural equation includes the above-mentioned control covariates (not shown in Fig. 1).
4 Results
4.1 CFA results
We assess the fit of the model through different indices, each measuring the fit in a different way. The chi-square test is significant (χ2 = 648 with 186 degrees of freedom, p < 0.000), which indicates that the model does not fit the data. However, the use of the chi-square statistic raises several issues in such a study, where the variables are not normally distributed and the large sample size is likely to lead to rejecting a good-fitting model. Together with the chi-square statistic, we comment the Standardized Root Mean Square Residual (SRMR; Asparouhov & Muthén 2018), Root Mean Square Error of Approximation (RMSEA; Steiger 1990), and Comparative Fit Index (CFI; Bentler 1990).
The value of these indices should be compared with a fixed cutoff, derived by Hu and Bentler (1999), that discriminates between a good and a bad model fit. Nevertheless, the fixed cutoffs have to be used with caution as many studies have stressed their lack of generalizability due to the fact that they may depend on some characteristics of the specific study, such as sample size, number of items, number of factors and factor reliability (McNeish and Wolf 2020).
The SRMR value (0.075) falls within the 0.08 threshold, which signals an acceptable fit whereas the RMSEA value (0.062) is a bit over the 0.06 suggested threshold for a good fit. The CFI value is equal to 0.923, with a value of 0.90 or greater suggesting an acceptable fit (Hox and Bechger 1999; Hu and Bentler 1999; Schermelleh-Engel et al. 2003).
As a whole, all the indices seem to point to an adequate model fit.
The standardised loadings (Table 3) are all significant and range from 0.474 for D15_i item (“Contribute money to the good school performance”) to 0.850 for D16_CI item (“Disruptive behaviours in the classroom”). Further, all three latent factors are positively and significantly correlated (p -value < 0.000).
We also calculated the omega coefficient (McDonald, 1999) to measure the internal consistency of the latent constructs (Dunn, et al. 2014; Bandalos 2018). We computed the nonlinear SEM reliability coefficient for ordered categorical items (Green and Yang 2009)Footnote 7. The value of omega index is 0.724 for the PI dimension (95% confidence interval: 0.675–0.768), 0.879 for PP (95% confidence interval: 0.851–0.900), and 0.900 (95% confidence interval: 0.878–0917) for SDB. The lowest reliability (0.724) may cause some concern, considering that more than 3/4 of the variability in the score on the latent factor PI can be attributable to random measurement errors. Nevertheless, values higher than 0.70 are considered to be acceptable by most standards.
4.2 Multilevel SEM results
The multilevel SEM with a random intercept was estimated. The results are illustrated in Fig. 2. Solid lines reflect significant paths whereas dashed lines represent non-significant paths. The reported coefficient estimates are in standardised units.
At the student level, the path from Socio-Economic Status to achievement is small and significantly positive, after controlling for gender, migration background and regularity. The standardised coefficient is 0.063 (p-values < 0.000) for reading and 0.057 (p-values < 0.000) for mathematics and the model accounts for 2.9% and 2.5% of the variance of the student level achievement for reading and mathematics, respectively.
At the school level, the model accounts for 25.6%, 9.9% and 23.3% of the variance of the latent constructs (PI, PP and SDB, respectively), whereas it explains 76.8% of the variance of the school-level achievement for reading and 72.4% for mathematics. We found partial support for H1 hypothesis on the direct effects of the latent constructs (PI, PP and SDB) on achievement. Only the path from SDB to reading and mathematics achievement is positive (the standardised coefficient for reading is equal to 0.158, for mathematics is 0.113) and significant (p-value < 0.000), which reveals that a better disciplinary climate in a school is significantly associated with a better school-level performance. Conversely, neither PI nor PP exerts significant effects on students performance, implying that parental involvement does not influence school-level outcomes.
H2 hypothesis requires to investigate whether, and to what degree, school-level variability in the latent constructs serves as a mediator of the school-level component of SES on school-level achievement. We found a significant simultaneous role of the multiple mediators in the relationship between SES and achievement. Indeed, in the face of an expected significantly positive direct effect of SES (the standardised path coefficient is equal to 0.580 for reading and 0.400 for mathematics), the results point to a total indirect positive effect through PI, PP and SDB (p-value = 0.001 for reading and p-value = 0.002 for mathematics). The total indirect effect arises from the positive and significant paths from SES to PI, PP and SDB (the corresponding standardised coefficients are 0.464, 0.256 and 0.383, respectively): it represents approximately 10% of the total effect. Nevertheless, we found that SDB only has a significant specific indirect effect (p-value = 0.000).
5 Conclusions
Empirical analyses, since the work by Coleman et al. (1966), have consistently demonstrated that wealthier students tend to gain better grades and, more generally, that the students’ socioeconomic characteristics are strong predictors of academic success.
Since the background factors (e.g., family poverty or immigrant status) are “given” and cannot be controlled by the schools, several studies have identified malleable policy factors to promote academic achievement. Among these malleable school factors, educational researchers have increasingly recognised the importance of school climate on students’ development . Indeed, “a large body of evidence connects a positive school climate to improvements in children’s learning and in healthy development in schools. A positive school climate is also an essential component within comprehensive school improvement processes.” (Berkowitz et al. 2017).
Although, in many countries, especially in the US, policymakers and educational leaders have invested large amounts of resources in developing services to improve school climate, in Europe and, particularly, in Italy, attention to this topic is rather rare. Therefore, we exploit the data from the National Institute for the Evaluation of the Education System (INVALSI) to assess the effect of some relevant aspects of school climate on academic performance of Italian students. Specifically, we analyse three latent constructs of school climate that account for the disciplinary behaviour of the students and two different aspects of parental involvement: the parents-teachers interactions and the parent participation to school activities and organization.
The findings from the estimation of a Multilevel Bayesian Structural Equation Model (MBSEM) suggest that policies aiming to prevent or reduce bullying and truancy behaviours among the students have direct implications for educational achievement; schools with better disciplinary climates score significantly higher when other characteristics (such as regional differences and differences in the composition by gender and migrant status) are considered. Furthermore, programmes that effectively cope with disruptive, intimidating and violent behaviours in the school can also impact educational equity; our findings show that the direct effect of socioeconomic status on school performance slightly decreases due to the mediation of school climate, which occurs, for the most part, through disciplinary behaviour.
These results show that the disciplinary dimension of school climate can be considered an important catalyst for academic success of Italian students.
In Italy, the debate on this topic is still at its initial stages; thus, investing effort and resources to provide focused data and empirical studies is imperative for policy development and school improvement even considering that climate-based policies do not require extensive and difficult structural reforms.
From this perspective, Ansary et al. (2015) describe and discuss some interesting examples (to be adapted and remodelled according to the characteristics of Italian school systems) adopted in Finland (KiVa project), Spain (SAVE Model), the United Kingdom (DFE Sheffield Anti Bullying Project) and Norway (Olweus Bullying Prevention Program).
Regarding the other investigated components of school climate (parent involvement), it seems that more involvement of parents does not have a significant effect on students’ achievement. Indeed, the direct effects on students’ performance both of the level of interaction between parents and teachers and of the participation of parents in school activities are statistically not significant. At the same time, their role as mediators of the SES level is also not significant. Although this result is consistent with several studies that assign a limited role to the involvement of parents (Domina 2005; Epstein 1991), nevertheless it must be interpreted in the light of the available data that do not allow to investigate in detail this dimension of the school climate. Indeed, we have no information on the extent of parental support at home (home-based involvement) but, at the same time, we are not able to measure and analyse the quality of parental involvement which certainly assumes greater importance than the simple quantity of it as suggested by Pomerantz et al., 2007.
Change history
25 August 2022
Missing Open Access funding information has been added in the Funding Note.
Notes
INVALSI is a public agency that operates under the supervision of the Italian Ministry of Education. Its task is to promote the improvement of educational attainment using national and international evaluations and to contribute to the development and growth of the Italian educational system.
Currently (school year 2020/21), the administration of national tests is mandatory in primary (2nd and 5th grades), lower secondary (8th grade) and upper secondary schools (10th grade).
We emphasise that these dimensions are investigated by analysing the school principal’s opinions. This constitutes a limitation of our study because principals might want to present their schools in a positive light. Furthermore, any heterogeneity across different classrooms of the same school is missing.
The index of Socio Economic Status (SES), provided by INVALSI, is used. This index takes into account parents’ occupations and education, along with variables that measure home possession goods (see Campodifiori et al. 2010 for details).
The geographical areas are defined as follows: North-West (including Liguria, Lombardia, Piemonte and Valle d’Aosta), North-East (including Emilia Romagna, Friuli-Venezia Giulia, Trentino-Alto Adige and Veneto), Center (including Lazio, Marche, Toscana, and Umbria), South (including Abruzzo, Campania, Molise, Puglia, Basilicata, Calabria, Sardegna and Sicilia).
The values of the indicators D16a to D16h have been reversed in such a way that higher values of the transformed indicators (D16ai to D16hi) are associated with a more positive student disciplinary behaviour.
To this purpose we have used the R packages lavaan (version 0.6-8), semTools (version 0.5-4) and MBESS (version 4.8.0).
References
Ansary NS, Elias MJ, Greene MB, Green S (2015) Best practices to address (or reduce) bullying in schools. Phi Delta Kappan 97:30–35. doi: https://doi.org/10.1177/0031721715610088
Antonakis J, Bendahan S, Jacquart P, Lalive R (2010) On making causal claims: A review and recommendations. Leadersh Q 21(6):1086–1120. doi: https://doi.org/10.1016/j.leaqua.2010.10.010
Asparouhov T, Muthén B (2010) Bayesian analysis using Mplus: Technical implementation. Muthén & Muthén, Los Angeles
Asparouhov T, Muthén B (2018) SRMR in Mplus. Retrieved from https://www.statmodel.com/download/SRMR2.pdf
Bandalos D (2018) Measurement theory and applications for the social sciences. Guilford
Bear GG, Yang C, Pell M, Gaskins C (2014) Validation of a brief measure of teachers’ perceptions of school climate: relations to student achievement and suspensions. Learn Environ Res 17:339–354. doi: https://doi.org/10.1007/s10984-014-9162-1
Benbenishty R, Astor RA, Roziner I, Wrabel SL (2016) Testing the Causal Links Between School Climate, School Violence, and School Academic Performance: A Cross-Lagged Panel Autoregressive Model. Educational Researcher 45(3):197–206. doi: https://doi.org/10.3102/0013189X16644603
Bentler PM (1990) Comparative fit indices in structural equation models. Psychol Bull 107:238–246
Berkowitz R, Iachini A, Moore H, Capp G, Astor RA, Pitner R, Benbenishty R (2017) School Climate. Oxford Research Encyclopedia of Education, Oxford Univeristy Press. DOI: https://doi.org/10.1093/acrefore/9780190264093.013.89
Borgonovi F, Montt G (2012) Parental Involvement in Selected PISA Countries and Economies, OECD Education Working Papers, No. 73, OECD Publishing, Paris, https://doi.org/10.1787/5k990rk0jsjj-en
Brand S, Felner R, Shim M, Seitsinger A, Dumas T (2003) Middle school improvement and reform: Development of validation of a school-level assessment of climate, cultural pluralism and school safety. J Educ Psychol 95:570–588. doi:https://doi.org/10.1037/0022-0663.95.3.570
Brand S, Felner RD, Seitsinger A, Burns A, Bolton N (2008) A large scale study of the assessment of the social environment of middle and secondary schools: the validity and utility of teachers’ ratings of school climate, cultural pluralism, and safety problems for understanding school effects and school improvement. J Sch Psychol 46:507–535. doi: https://doi.org/10.1016/j.jsp.2007.12.001
Buchmann C, Parrado EA (2006) Educational achievement of immigrant-origin and native students: a comparative analysis informed by institutional theory. In: Baker DP, Wiseman AW (eds) The Impact of Comparative Education Research on Institutional Theory. Emerald Group Publishing Limited, Bingley, pp 335–366
Burcu S, Sungur S (2009) Parental influences on Students’ Self-concept, Task Value Beliefs, and Achievement in Science. Span J Psychol 12:106–117. doi: https://doi.org/10.1017/s1138741600001529
Campodifiori E, Figura E, Papini M, Ricci R (2010) Un indicatore di status socio-economico culturale degli allievi della quinta primaria in Italia. INVALSI Working Paper, 2
Castellano R, Longobardi S (2020) The Long March Towards School and Student Assessment in Italy. In: Harju-Luukkainen H, McElvany N, Stang J (eds) Monitoring Student Achievement in the 21st Century. Springer, Cham, pp 181–194 https://doi.org/10.1007/978-3-030-38969-7_15.
Clifford M, Menon R, Gangi T, Condon C, Hornung K (2012) Measuring school climate for gauging principal performance: A review of the validity and reliability of publicly accessible measures. American Institutes of Research, Washington, DC
Cohen J (2006) Social, emotional, ethical, and academic education: Creating a climate for learning, participation in democracy, and well-being. Harv Educational Rev 76:201–237. doi: https://doi.org/10.17763/haer.76.2.j44854x1524644v
Coleman JS, Campbell EQ, Hobson CJ, Mcpartland J, Mood AM, Weinfeldand FD, York RL (1966) Equality of Educational Opportunity, Washington, DC, USA
Domina T (2005) Leveling the Home Advantage: Assessing the Effectiveness of Parent Involvement in Elementary School. Sociol Educ 78:233–249. doi: https://doi.org/10.1177/003804070507800303
Dunn TJ, Baguley T, Burnsden V (2014) Alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. Br J Psychol 105:3: 399–412
El Nokali N, Bachman H, Votruba-Drzal E (2010) Parent Involvement and Children’s Academic and Social Development in Elementary School. Child Dev 81(3):988–1005. doi: https://doi.org/10.1111/j.1467-8624.2010.01447.x
Epstein J (1991) Effects on Student Achievement of Teachers’ Practices of Parent Involvement. Adv Reading/Language Res 5:261–276
Fan X (2001) Parental involvement and students’ academic achievement: a growth modeling analysis. J Experimental Educ 70(1):27–61. doi: https://doi.org/10.1080/00220970109599497
Giambona F, Porcu M (2018) School size and students’ achievement. Empirical evidences from PISA survey data. Socio-Economic Plann Sci. 64(C):66-–77 doi:https://doi.org/10.1016/j.seps.2017.12.007
Green SB, Yang Y (2009) Reliability of summed item scores using structural equation modeling: An alternative to coefficient alpha. Psychometrika 74(1):155–167. doi:https://doi.org/10.1007/s11336-008-9099-3
Guo P, Choe J, Higgins-D’Alessandro A (2011) Report of construct validity and internal consistency findings for the comprehensive school climate inventory. Unpublished manuscript. Fordham University, Bronx, NY
Güzel ÇI, Berberoğlu G (2005) An analysis of the Programme for International Student Assessment 2000 (PISA 2000) mathematical literacy data for Brazilian, Japanese, and Norwegian students. Stud Educational Evaluation 31:283–314. doi: https://doi.org/10.1016/j.stueduc.2005.11.006
Halpin AW, Croft DB (1963) Organizational climate of schools. Midwest, Chicago Administration, Center University of Chicago
Harris A, Chrispeels JH (eds) (2006) Improving schools and educational systems: International perspectives. Routledge
Hox J, Bechger T (1999) An Introduction to Structural Equation Modeling. Family Sci Rev 11:354–373
Hoy WK, Hannum J, Tschannen-Moran M (1998) Organizational climate and student achievement: A parsimonious and longitudinal view. J School Leadersh 8:336–359
Hoy WK, Smith PA, Sweetland SR (2002) The development of the organizational climate index for high schools: Its measure and relationship to faculty trust. The High School Journal 86(2):38–49. doi:https://doi.org/10.2307/40364336
Hu L, Bentler P (1999) Cutoff criteria for fit indices in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model 6:1–55. doi: https://doi.org/10.1080/10705519909540118
Jeynes WH (2012) A Meta-analysis of the Efficacy of Different Types of Parental Involvement Programs for Urban Students. Urban Educ 47(4):706–742. doi: https://doi.org/10.1177/0042085912445643
Kyriakides L, Creemers BPM (2008) Using a multidimensional approach to measure the impact of classroom level factors upon student achievement: a study testing the validity of the dynamic model. School Eff School Improv 19:183–205. doi: https://doi.org/10.1080/09243450802047873
Longobardi S, Falzetti P, Pagliuca MM (2018) Quis custodiet custodes? A multistep approach to detect and correct cheating in Italian student data. Stat Methods Appl. 27:515–543https://doi.org/10.1007/s10260-018-0426-2
Ma X, Willms JD (2004) School disciplinary climate: characteristics and effects on eight grade achievement. Alta J Educ Res 50:169–188
Ma X, Jong C, Yuan J (2013) Exploring reasons for the East Asian success in PISA, in H.-D. Meyer & A. Benavot (Eds.), PISA, power, and policy: The emergence of global educational governance, 225–246, Oxford: Symposium Books
MacNeil AJ, Prater DL, Busch S (2009) The effects of school culture and climate on student achievement. Int J Leadersh Educ 12:73–84. doi:https://doi.org/10.1080/13603120701576241
Marsh H, Lüdtke O, Nagengast B, Trautwein U, Morin A, Abduljabbar A et al (2012) Classroom climate and contextual effects: Conceptual and methodological issues in the evaluation of group-level effects. Educational Psychol 47:106–124
McDonald RP (1999) Test theory: A unified treatment. Lawrence Erlbaum, Mahwah, NJ
McNeish D, Wolf MG (2020) Dynamic fit index cutoffs for Confirmatory Factor Analysis models, PsyArXiv
Muthén BO, Asparouhov T (2008) Growth mixture modeling: Analysis with non-Gaussian random effects. In: Fitzmaurice G, Davidian M, Verbeke G, Molenberghs G (eds) Longitudinal data analysis. Chapman & Hall/CRC, Boca Raton, FL, pp 143–165
Muthén B, Asparouhov T (2012) Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychol Methods 17(3):313–335. https://doi.org/10.1037/a0026802
Niederle M, Vesterlund L (2010) Explaining the Gender Gap in Math Test Scores: The Role of Competition. J Economic Perspect 24(2):129–144. doi: https://doi.org/10.1257/jep.24.2.129
Ning B, Van Damme J, Van Den Noortgate W, Yang X, Gielen S (2015) The influence of classroom disciplinary climate of schools on reading achievement: a cross-country comparative study. School Eff School Improv 26(4):586–611. doi: https://doi.org/10.1080/09243453.2015.1025796
OECD (2010) PISA 2009 Results: What Makes a School Successful?, Resources, Policies and Practices. Volume IV)
OECD (2019) PISA 2018 Results (Volume III): What School Life Means for Students’ Lives. PISA, OECD Publishing, Paris. https://doi.org/10.1787/acd78851-en
Patton GC, Bond L, Carlin JB, Thomas L, Butler H, Glover S, Bowes G (2006) Promoting social inclusion in schools: A group-randomized trial of effects on student health risk behavior and well-being. Am J Public Health 96:1582–1587. doi: https://doi.org/10.2105/AJPH.2004.047399
Pearl J (2012) The Causal Foundations of Structural Equation Modeling. In R.H.Hoyle (Ed.), Handbook of Structural Equation Modeling. Guilford Press, New York, pp 68–91
Perry A (1908) The management of a city school. Macmillan, New York, NY
Pomerantz EM, Moorman EA, Litwack SD (2007) The how, whom and why of parents’ involvement in children’s academic lives: More is not always better. Rev Educ Res 77:373–410. doi:https://doi.org/10.3102/003465430305567
Preacher KJ, Zyphur MJ, Zhang Z (2010) A general multilevel SEM framework for assessing multilevel mediation. Psychol Methods 15:209–233
Pyper JR, Freiberg HJ, Ginsburg M, Spuck DW (1987) Instruments to measure teacher, parent, and student perceptions of school climate. in: Phi Delta Kappa, Bloomington
Quintano C, Castellano R, Longobardi S (2012) The effects of socioeconomic background and test-taking motivation on Italian students’ achievement, in A. Di Ciaccio, M. Coli, J.M. Angulo, Advanced Statistical Methods for the analysis of large data-sets, series Studies in Theoretical and Applied Statistics, Springer: Berlin
Rangvid BS (2007) School composition effects in Denmark: Quantile regression evidence from PISA 2000. Empirical Economics 33:359–388. doi: https://doi.org/10.1007/s00181-007-0133-6
Schermelleh-Engel K, Moosbrugger H, Müller H (2003) Evaluating the Fit of Structural Equation Models: Tests of Significance and Descriptive Goodness-of-Fit Measures. Methods of Psychological Research, 8(2), 23–74
Sherblom SA, Marshall JC, Sherblom JC (2006) The relationship between school climate and math and reading achievement. Journal of Research in Character Education, 4(1–2):19–31
Sheridan SM, Bovaird JA, Glover TA, Garbacz SA, Witte A, Kwon K (2012) A Randomized Trial Examining the Effects of Conjoint Behavioral Consultation and the Mediating Role of the Parent–Teacher Relationship. School Psychol Rev 41(1):23–46. doi: https://doi.org/10.1080/02796015.2012.12087374
Sheridan SM, Witte AL, Holmes SR, Couuts MJ, Dent AL, Kunz GM, Wu C (2017) A randomized trial examining the effects of conjoint behavioral consultation in rural schools: Student outcomes and the mediating role of the teacher–parent relationship. J Sch Psychol 61:33–53. doi: https://doi.org/10.1016/J.JSP.2016.12.002
Shin J, Lee H, Kim Y (2009) Student and school factors affecting mathematics achievement: International comparisons between Korea, Japan, and the USA. School Psychol Int 30:520–537. doi: https://doi.org/10.1177/0143034309107070
Simons-Morton BG, Crump AD (2003) Association of parental involvement and social competence with school adjustment and engagement among sixth graders. J Sch Health 73(3):121–126. doi: https://doi.org/10.1111/j.1746-1561.2003.tb03586.x
Sink CA, Spencer LR (2007) Teacher version of the My Class Inventory-Short Form: An accountability tool for elementary school counselors. Professional School Counseling, 11, 129–139. doi:1.5330/PSC.n.2010-11.129
Steiger JH (1990) Structural model evaluation and modification: An interval estimation approach. Multivar Behav Res 25:173–180
Stewart EB (2008) School structural characteristics, student effort, peer associations, and parental involvement: The influence of school- and individual-level factors on academic achievement. Educ Urban Soc 40:179–204. doi: https://doi.org/10.1177/0013124507304167
Voight A, Austin G, Hanson T (2013) A climate for academic success: How school climate distinguishes schools that are beating the achievement odds. WestEd, San Francisco
Wang M, Sheikh-Khalil S (2014) Does parental involvement matter for student achievement and mental health in high school? Child Dev 85/2:610–625. https://doi.org/10.1111/cdev.12153
Zellman GL, Waterman JM (1998) Understanding the impact of parent school involvement on children’s educational outcomes. J Educational Res 91(6):370–380. doi: https://doi.org/10.1080/00220679809597566
Funding
Open access funding provided by Università Parthenope di Napoli within the CRUI-CARE Agreement.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Longobardi, S., Pagliuca, M.M. & Regoli, A. School climate and academic performance of Italian students: the role of disciplinary behaviour and parental involvement. Stat Methods Appl 31, 1355–1373 (2022). https://doi.org/10.1007/s10260-022-00632-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10260-022-00632-7