Italian students’ performance and regional decomposition

We relate students’ math scores in the OECD-PISA test to school characteristics. The average math score for Italian students has been increasing in 2009. The determinants of this growth are analyzed by the Oaxaca–Blinder decomposition, that is particularly useful in comparing groups. The progress in educational attainments shows a different composition between northern and southern schools. In the North-Center regions, improvements are explained by school endowments, while in the South they are also driven by external factors that are not explained by the estimated model and are linked to improvements in students' attitude to education leading to a more favorable disciplinary climate. The regional gap decreases but does not disappear.


Introduction
There are many tests aiming to detect students' performance, like the US accountability tests, the British Cohort Study, or the Australian NAPLAN program. They provide data for educational monitoring as input to achieve reliable literacy rates and enables a sensible comparison across countries with a focus on resources allocation and policymaking of institutions and funding agencies (UNESCO 2006). Cross-national studies, by disclosing disparities in school resources, school organization, and teaching practices, encourage investments in education (Postlethwaite 2004). 1 Indeed, students' performance can be regarded as an indicator of economic growth, and schooling is important for boosting the rate of growth of the economy (Barro 1997). For instance, mathematical skills are considered a particularly good proxy for labor force quality. For instance, Hanushek (2006) 1 3 relates a one standard deviation difference in students' math performance to a 1 percent difference in annual per capita GDP growth rates.
There are, of course, mixed feelings about the effectiveness of those cross-countries assessment programs. According to Breakspear (2012), PISA methodologies have been embedded within national systems to evaluate school organization, and large-scale evaluation has been introduced for systems that did not have any previous national assessment. The PISA test has complemented national data and has promoted the comparison of national results against international benchmarks (Sellar and Lingard 2018). Mulongo and Amod (2019), for instance, analyzing Kenya, Tanzania, and South Africa policies, find that at least 11 policy/strategic documents formulated between the years 2000 and 2015 respond to recommendations or findings emanating from the cross-national learning assessments.
To the contrary, other studies criticize cross-countries assessment programs suggesting that, to emulate the PISA performance of the 'best education systems', important contextual and cultural differences existing within and between education systems are often ignored or sidelined (Harris and Jones 2017). In other words, the same strategy implemented in very different contexts results in highly variable outcomes and impact. Harris and Jones (2018), analyzing the school structure in seven different countries (Australia, England, Indonesia, Hong Kong, Malaysia, Russia, and Singapore), conclude that explanations of performance differentials need much more attention and acknowledgement of cultural and contextual influences: The context in which policy is implemented requires great consideration and understanding.
In addition, other authors are concerned about the occurrence of non-standard gradings approaches. They show variations across macro-geographical areas in the degree of strictness in teachers' standards. These studies find empirical evidence that teachers' school marks and students' performance in the standardized tests are not homogenous across geographical areas, with a focus on the North-Center versus South divide in Italy (Argentin and Triventi 2015;Battistin et al. 2017). Diverging grading standards provide differing information about the student's competencies and imply diverse opportunities between students who attend school in different regions.
In what follows, the scores of math test in the international OECD-PISA questionnaire for 15-year-old Italian students are analyzed. The OECD-PISA is a cross-national learning assessment program aiming to gain information on countries educational outcomes in support to policymakers (Knight et al. 2012). In this analysis, PISA test scores are related to explanatory variables describing school characteristics such as size, funding, student-teacher ratio, to link student performance and school effectiveness. Our focus is on the link between student performance and school environment in a setting characterized by a marked regional divide, with Italian Southern regions lagging.
We look at the evolution of this link both over time and across regions, to seek the determinants of scores improvements and to find the best devices to close the educational gap across Italian regions. The analysis particularly focuses on regional discrepancies by implementing a decomposition approach that provides fruitful insights wherever a divide occurs. Disparities between high/low developed regions, or between urban versus rural areas, or different school's tracks, can be fruitfully analyzed by decomposition. The focus of the Oaxaca-Blinder (1973) decomposition is to point out the source of the discrepancy. Is it due to a difference in the variables of the model across regions/ areas/schools' organization, or is it due to external factors not comprised in the equation? In the former case, the discrepancy is explained by a difference in endowments, that is in the covariates of the model. When this is the case, the policy effort to close the gap attempts to increase endowments of one of the groups and assessing which factor has a wide impact on students' scores helps selecting the most appropriate intervention. Vice versa, a discrepancy due to external factors does not provide any potential factor of policy interest. The Italian data set provides a valid example on how to investigate within country discrepancies. The decomposition approach allows to identify which components could be changed in order to improve the general attainments.

Schools endowments
There is a growing interest among academics and policymakers on school effectiveness research. The central hypothesis of this research area postulates that certain characteristics of school impact student educational attainment, and this impact holds even after controlling for the students' socioeconomic, academic, and demographic characteristics (Goldstein and Woodhouse 2000;Opdenakker and Van Damme 2001). The school effect, once the socioeconomic level of the students has been controlled for, ranges between 10 and 30%; it is higher in mathematics than in either languages or science, and it is also higher in primary education than in secondary education. It appears that the most important source of school inefficiency is the resource endowment. However, there are mixed results on the impact of specific educational elements. Kruger (1999) in the STAR (Student-Teacher Achievement Ratio) experiment, where students are randomly assigned to classes differing in size, concludes that students of small size classes yield better performances, and later on, by adding college records (Kruger and Whitmore 2001), he finds that students from small classes are more likely to apply to college, particularly so for Afro-American. Vice versa, Hoxby (2000) does not find any significant effect of class size on student achievements, together with Woessman and West (2002) who analyze TIMMS (Third International Mathematics and Science Study) data. Class size is not the only one element of school endowments having an ambiguous impact on school attainment. Student-teacher ratio, library facilities, books availability, and funding are in turn positively or negatively related to student performance or statistically non-significant. In the literature there are, indeed, mixed findings so that rise in school inputs does not necessarily imply students improved scores. Barro and Lee (2001), looking at scholastic districts, find a positive impact of school endowments on student scores in internationally comparable tests. Kruger (1992, 1996) find a positive impact of school resources as well, in terms of both (i) students staying longer at school and thus (ii) receiving afterward higher returns to education. They find that an average 10% increase in educational expenditure per student yields a 1-2% increase in future annual earnings. On the other hand, Hanushek (1996) finds a stable student performance against 3.5% annual increase in US educational expenditure, while Gundlack et al. (2001) in the OECD countries find a decline in student attainment, annually by a 2-4%. Interestingly, Fiske and Ladd (2000) take advantage of school reforms in New Zealand that shifts from a centralized to a decentralized school system occurred in 1991. They find an increased polarization of good students in the best schools, with the low performing schools generally located in deprived areas. In what follows, the links between school endowments and students score are analyzed, particularly focusing on regional discrepancy in the school system.

3 3 The PISA test and the Italian school
The OECD-PISA test of student proficiency was administered since year 2000. 2 The math scores of 15-year-old Italian students have been slightly above the OECD average score in 2009, 495 versus the OECD average of 491. In earlier years, the average Italian score was below average, although showing an improving trend over time, and it declined again below average in 2012. 2 According to the OECD country note, 2 Italy's mean performance improved between 2003 and 2012 by an average of 20 score points, moving substantially closer to the OECD average… Most of the improvement in mathematics performance was observed between 2006 and 2009… Italy is one of the fastest improving countries in mathematics performance among those countries that participated in every PISA assessment since 2003… Italy shows above-OECDaverage equity in education outcomes, with 10% of the variation in student performance in mathematics attributable to differences in student socio-economic status… While performance improved, equity remained stable… The improvement in mathematics performance is observed among all socio-economic groups: disadvantaged students improved by 27 score points and advantaged students by 17 score-points.
This suggests focusing on the changes occurred in 2009 to examine its main determinants. We relate the OECD-PISA test math score achieved by Italian students, y, to the explanatory variables describing school characteristics: field; school size; funding; student-teacher ratio; library facilities, number of computers, lab equipment, and textbooks. This allows to estimate the link between student performance and school environment.
As mentioned, growth analysis emphasizes school attainment and shows its strong links to differences in economic growth across countries. Mathematical skills are considered a particularly good proxy for labor force quality, and Hanushek (2006) relates a one standard deviation difference in students' math performance to a 1 percent difference in annual per capita GDP growth rates.

The data set
The estimated equation explains math test scores, y, as a function of funding represented by a dummy variable, private, which assumes unit value for private schools which in Italy are remedial and very few in number; school track which uses the dummy variables academic and technical to distinguish vocational fields; a dummy for gender which takes a unit value for boys who generally perform better than girls in math but worse in reading proficiency; school size a variable for the number of students enrolled; number of computers in the school; student-teacher ratio; proportion of certified teachers in the school; shortage of teachers; four categorical variables for poor library facilities, shortage of computers, shortage of textbooks, shortage of lab equipment. These four variables are coded as: a lot, some, a little, none at all. Each of them is transformed into a numerical variable assigning values ranging from 1 (none at all) to 4 (a lot). 3 The sample comprises n = 52,067  The bottom sections of Table 1 collect the summary statistics for the explanatory variables. The curricula show a steady increase over time of students enrolled in the academic track while vocational and technical track decrease steadily. The variables measuring shortage in lab equipment, in textbook, and in library facilities do not show any improvement over time.  Table 2. The math score test y is explained by academic, technical, boys, school size, number of computers, student-teacher ratio, proportion of certified teachers, private, shortage of library Almost all the estimated coefficients are statistically significant. School size, proportion of certified teachers, private schools together with the poor facilities coefficients (shortage of textbook, shortage of lab equipment) have a negative impact on students' scores. Comparison over time shows a sizable reduction in year 2009 of the math gender gap and an improvement in the technical track coefficient. Shortage of lab equipment and proportion of certified teachers are not statistically significant in 2009.

Decomposition over time
Based on the selected covariates of the math scores equation, the changes over time reported in Table 2 can be decomposed into covariates and coefficients effect by the Oaxaca (1973) and Blinder (1973) decomposition approach. Other authors have implemented a decomposition approach with PISA scores. For instance, Gigena et al. (2011) analyze the discrepancy in PISA reading scores between Chile, Mexico, and Argentina. They show how differently covariates and coefficients effects perform in the three countries. In Mexico, the decrease in covariates is balanced by the increase in coefficients. Chile shows a positive effect in both endowment and coefficients, while the opposite occurs for Argentina. Barrera-Osorio et al. (2011) consider PISA math scores in Indonesia, finding a key role for the improved performance in an adequate teacher supply.
We begin our analysis by comparing data for years 2000-2003-2006, that are indexed 0, with the data for year 2009 indexed 1, namely ( y 0 − y 1 ). Consider the comparison between y 0 and y 1 on average, E(y 0 − y 1 ). By adding and subtracting the term Ex 1 ̂0 and by rewriting Ey 0 = Ex 0 ̂0 and Ey 1 = E x 1 ̂1 , the average difference can be written as The term Ex 1 ̂0 is the so-called counterfactual, an unobservable term that computes how would y 1 be in case x 1 would have the same impact as group 0, impact measured by ̂0 . Thus, the counterfactual Ex 1 ̂0 measures the effect of the group 1 covariates, i.e., their recent value, evaluated at the group 0 estimated coefficients, i.e., their past coefficients. The decomposition breaks the average difference y 0 − y 1 into changes due to coefficients, Ex 1 ( ̂0 − ̂1 ), and changes explained by the covariates, (Ex 0 − Ex 1 )̂0. A change in the covariates (Ex 0 -Ex 1 ) signals a variation in math scores explained by group differences, in this case explained by changes in schools' resources. This provides a potential factor of policy interest. Vice versa, a change in the estimated coefficients ( ̂0 − ̂1 ) represents a change in students' performance that is not explained by the covariates of the equation and that is outside policy intervention. This unexplained part is often used as a measure for discrimination. 6 More in general, it computes the effects of group differences in unobserved predictors.
In addition, an interaction term, (Ex 0 -Ex 1 ) ( ̂0 − ̂1 ), can be added to account for simultaneous differences in endowments and coefficients: The estimates of the above decomposition are reported in the first column of Table 4. 7 The results show an overall improvement in students' scores over time that is sizable and statistically significant: The difference Ey 0 − Ey 1 = − 20.08 (se = 0.775) tells of a general significant improvement in year 2009. This improvement is due to endowments, with schools' characteristics accounting for the greatest part (Ex 0 − Ex 1 )̂0 = − 15.04 (se = 1.02), while the increase in the coefficients effect is relatively small, Ex 1 ( ̂0 − 1 ) = − 4.69 (se = 0.888). The interaction term is not statistically significant, with an increment of (Ex 0 − Ex 1 ) ( ̂0 − ̂1 ) = − 0.343 (se = 1.11). School characteristics improved in 2009, and the unexplained change due to the coefficients is comparatively small. The first column of Table 5 shows for each explanatory variable the detailed contribution of each covariate and of each coefficient to the global improvement occurred in year 2009. Academic, computer, boys, private schools are among the endowments that contributed most to the increased performance in year 2009. The coefficients of technical schools, student-teacher ratio, shortage of lab equipment, shortage of textbooks, shortage of teachers, computers and proportion of certified teachers have all improved their coefficient effect in 2009 regardless of their endowment. The interaction term, that overall is not statistically significant, is significant for many of the variables of the model individually considered. At times, the interaction term reinforces the improvement signaled by covariates and coefficients effects, at times mitigates it.

Regional inequality
The OECD country notes 2 state that: In Italy more than half (51.7%) of the overall variation in student performance lies between schools: this means that two students who attend different schools can be expected to perform at very different levels. The comparatively large between-school variation in performance to an extent reflects the large regional differences in performance which can be observed in Italy, although large between-school differences can be observed even when regional differences are considered. Therefore, we look at the regression explaining math scores in each macro region, North-Center and South. Table 3 collects the OLS estimates for the entire time period, 2000-2009. The North-Center regions perform generally better than the southern ones, as shown by the greater coefficients of academic and technical truck, proportion of certified teachers, by the reduced gender gap in math scores, by the nonsignificance of shortage of lab equipment and by the milder impact of private schools and shortage of textbook.
A deeper analysis is offered by the regional Oaxaca-Blinder decomposition, comparing the time period 2000-2006, indexed 0, and year 2009, indexed 1, in the South and independently in the North-Center. Once again, the difference in math scores is decomposed into covariates, coefficients, and interaction effects by implementing the Oaxaca-Blinder decomposition separately in each region. 8   The impact of the covariates/coefficients/interaction terms for the southern regions over time is reported in Table 4 column 2. The third column of this table collects the results for the North-Center regions over time.
In sum, the regional decomposition provides different results for the North-Center and the southern regions. In the North-Center, the covariates, i.e., the school characteristics, show a significant improvement over time, which is largely curbed by the worsened impact of coefficients and interaction effects. In the South, the positive changes are driven by all the three terms: covariates, coefficients, and interaction effects. As mentioned, the coefficients effects refer to aspects beyond the schools' control. Among the many possible explanations for these results is students' attitude to education. The OECD country note 2 reports that: Between 2003 and 2012, the disciplinary climate in Italian schools improved significantly. In 2003, 39% of students reported that, in most or all lessons, the teacher has to wait a long time for students to quiet down; by 2012 that proportion had decreased to 31%. Similarly, in 2003, 42% of students reported that there is noise and disorder in most or all lessons. By 2012 this percentage had decreased to 36%.
The different behavior in the decomposition within regions confirms that both temporal and regional discrepancies should be analyzed.
The second and third columns of Table 5 detail the impact on the decomposition of each variable of the model. By this set of results, it is possible to find which covariate improves and which one worsens over time within each region and which variable provides a fruitful device to reduce the regional gap (or to achieve whatever other relevant target for policymakers). Computer, teacher shortage, computer shortage and school size are the endowments that most improved over time in the southern regions, while the student-teacher ratio sizably worsened. The former group of variables provides good tools to improve southern regions educational attainment and to reduce the educational gap across regions. The widest significant coefficients effect is for technical track, student-teacher ratio, and proportion of certified teachers, but these coefficients effects cannot be considered factors of policy interest since they are beyond school control.
In the North-Center group of schools, the greatest endowment increase is in academic track, private schools, and computers, but they are greatly offset by the sizable worsening in 2009 of the coefficient effects for proportion of certified teachers and school size.
Finally, the above results show that over time the regional gap has been reduced, although not completely closed, and a greater improvement in southern schools' characteristics would be very helpful.

Overall results
The analysis of the behavior over time and with respect to differing regional characteristics provides the following results: (i) The average Italian math score has been growing over time: In 2000  but also to compare North-Center versus southern regions. While changes in scores over time in the North-Center are related to improvements in the explanatory variables, i.e., explained by improvements in school characteristics, in southern schools the recent improvements in math scores are due to changes in the covariates and even more in the coefficients. The coefficients include effects which modify the conditional distribution of student performance and which are not directly related to school characteristics. They may be linked, for instance, to an improved attitude among students toward education. (iii) Unfortunately, the southern regions improvements do not close the regional gap.
These results suggest that, by complementing the southern coefficients effect with an improvement in southern schools' characteristics, the regional educational gap could be further reduced.

Conclusions
Among the many tests aiming to detect students' performance, like the US accountability tests, the British Cohort Study, or the Australian NAPLAN program, the PISA crossnational learning assessment program is the one here considered. The analysis focuses on Italian students' scores in math with its sizable enhancement in year 2009. The Italian peculiarity is a marked regional divide that makes interesting to investigate the source of the improvement. We focus on student performance linked to school characteristics rather than socioeconomic variables. The average math scores in the OECD-PISA test in Italy have been growing, particularly in year 2009, to overtake the OECD average. The determinants of this growth are here analyzed and decomposed into terms explained and unexplained by the schools' features selected in the estimated model, respectively, the covariate and coefficient effects. A change in covariates mirrors the change in the impact of school characteristics, while a change in the coefficients shows a variation that cannot be ascribed to school characteristics. The decomposition analysis is generally implemented to analyze groups discrepancy like regional divide, urban versus rural gap, or, for instance, academic versus technical schools' attainments. It allows to point out potential factors of policy interest to curb/enhance discrepancies. The analysis here implemented considers both temporal and regional changes, the latter linked to the Italian regional divide that is fully reflected in the educational attainments.
The temporal decomposition shows a large improvement mostly due to the covariates, that is to increased schools' endowment over time. The regional decomposition shows differing behaviors. Over time, the northern regions present a steady improvement in the covariates (endowment) largely curbed by the coefficients effect. The southern schools, instead, show an improvement in both covariates and coefficients. Thus, the rise in students' math scores is due to a higher impact of the school covariates nationwide, particularly sizable over time, and to improvements in the southern school coefficients, i.e., to changes in characteristics outside the schools' control.
While a change in the covariates is directly attributable to the school environment, the change in the coefficients is less straightforward. A possible explanation lies in variables such as students' attitude to education which has improved greatly in recent years. The latter is measured by variables such as the number of skipped classes that are generally beyond the school's control.
The different behavior in the decomposition within regions confirms that both temporal and regional discrepancies should be analyzed. The combined regional and temporal analysis shows that the regional gap occurring in the 2000-2003-2006 period is reduced in 2009, but there is still a North-Center versus South educational gap.
The finding that southern student improvements in math scores are partly related to school characteristics leaves room for interventions. Improvements in southern school covariates could reduce the regional divide at least in the educational attainment.
Funding Open Access funding provided by Università degli Studi di Napoli Federico II.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creat iveco mmons .org/licen ses/by/4.0/.