The Case for Good Discipline? Evidence on the Interplay Between Disciplinary Climate, Socioeconomic Status, and Science Achievement from PISA 2015

Abstract

In both educational and psychological research, the relation between socioeconomic status (SES) and academic achievement is the most widely examined contextual effect. While several research syntheses have reported evidence of positive and significant SES–achievement relations (i.e., higher SES is associated with better academic achievement in several domains), they also reported substantial variation across educational contexts, such as classrooms, schools, and educational systems, and proposed mechanisms underlying these relations. This chapter addressed this variation and tested three hypotheses on the interplay between socioeconomic status, the disciplinary climate in science lessons, and science achievement—the compensation hypothesis, the mediation hypothesis, and the moderation hypothesis. Utilizing the Programme for International Student Assessment (PISA) 2015 data from the Nordic countries (Denmark, Finland, Iceland, Norway, and Sweden), multilevel structural equation modeling provided evidence to test the contextual, indirect, and cross-level interaction effects. While evidence for the compensation hypothesis existed in most Nordic countries, evidence supporting the mediating and moderating roles of the disciplinary climate for the SES–achievement relation was sparse.

Good classroom discipline, an orderly learning environment, and few disruptions of instruction are considered prerequisites for a good school climate and instructional quality. While most of the extant research has been concerned with establishing that a disciplinary climate—a climate that requires the definition of desirable student behaviors and the prevention of undesirable student behaviors (Hochweber, Hosenfeld, & Klieme, 2014)—is significantly related to academic achievement (Berkowitz, Moore, Astor, & Benbenishty, 2017), less effort has been made to establish this relation in the context of equity or equality (Atlay, Tieben, Hillmert, & Fauth, 2019). Specifically, moving beyond merely describing the socioeconomic status (SES)–achievement relation as an indicator of (in-)equality, researchers and policy makers have become more and more interested in studying the following: (a) the extent to which a disciplinary climate may compensate for the effect of SES on academic achievement, (b) the mechanisms behind the relations among SES, achievement, and disciplinary climate, and (c) the extent to which the disciplinary climate may decrease possible achievement gaps between students of different SES (Berkowitz et al., 2017; Ning, Van Damme, Van Den Noortgate, Yang, & Gielen, 2015). However, the body of evidence clarifying the role disciplinary climate plays for SES, academic achievement, and the SES–achievement relation is diverse. For instance, while some evidence suggests that a disciplinary climate is directly related to achievement above and beyond SES (Bellens, Van Damme, Van Den Noortgate, Wendt, & Nilsen, 2019), some evidence suggests that it may mediate the relation between SES and achievement (Liu, Van Damme, Gielen, & Van Den Noortgate, 2015). Some further evidence suggests that a good disciplinary climate moderates the SES–achievement relation (Ning et al., 2015). This diversity in the nature of the relations among SES, disciplinary climate, and academic achievement ultimately results in different interpretations of the role disciplinary climate plays: While some researchers may conclude that a good disciplinary climate is related to better achievement independent of students’ or schools’ SES, others may conclude that a good disciplinary climate is more likely to occur in high-SES schools, resulting in better achievement. Finally, other researchers may conclude that a good disciplinary climate is associated with smaller achievement gaps—in other words, in schools with a good disciplinary climate, the achievement differences are hardly retraceable to SES differences. In the extant literature, these three perspectives have been summarized in three hypotheses—namely, the compensation, mediation, and moderation hypotheses (Berkowitz et al., 2017). Through the lenses of these hypotheses, the Programme for International Student Assessment (PISA) 2015 data of the five Nordic countries (Denmark, Finland, Iceland, Norway, and Sweden) were analyzed, and the evidence base for or against a compensation, mediation, or moderation mechanism describing the relations among SES, disciplinary climate, and science achievement was examined. Ultimately, the resultant evidence could clarify the role of disciplinary climate for SES, achievement, and the SES–achievement relation for the PISA 2015 Nordic country data and highlight plausible conclusions that could be drawn in the context of equity and equality. Following the framework proposed by Willms and Tramonte (2019), this study considers the relation between SES and disciplinary climate an indicator of equity (i.e., representing possible differences in the opportunities to access a good disciplinary climate in school science lessons), while the relation between SES and science achievement is seen as an indicator of equality (i.e., representing possible differences in educational outcomes). These conceptualizations resonate with the “equality–equity model” proposed by Espinoza (2007), which can be characterized as follows: (a) possible SES differences in disciplinary climate may represent differences in access to education, or more precisely, access to the same quality of education to address basic educational needs; and (b) possible SES differences in science achievement (i.e., educational achievement based on test performance in the dimension of “output”) represent inequalities for students across social groups.

1 Theoretical Framework

1.1 Disciplinary Climate and Academic Achievement

The disciplinary climate represents one of the most extensively studied aspects of schooling and instruction (Atlay et al., 2019; Seidel & Shavelson, 2007). Although a plethora of conceptualizations exist, the extant body of literature seems to converge in that the disciplinary climate represents a climate in schools and/or classrooms that requires the identification of desirable and the prevention of undesirable student behaviors (Hochweber et al., 2014). This conceptualization clearly goes beyond strategies to handle disruptive behavior in educational settings (Atlay et al., 2019) and comprises instructional approaches, such as setting and communicating classroom rules, establishing routines, providing an orderly and functional classroom or school setting, monitoring school and/or classroom activities, and intervening if necessary (e.g., Hochweber et al., 2014; Seidel & Shavelson, 2007). To add to the complexity, teachers must adapt these approaches to the specific classroom or school contexts, especially in socially diverse settings with substantial variation in SES or minority status (Emmer & Stough, 2001; Rjosk et al., 2014). In this sense, establishing a good disciplinary climate is considered part of teacher competence, and the instructional approaches taken to accomplish it is part of instructional quality (Lipowsky et al., 2009; Seidel & Shavelson, 2007). Despite this anchoring in the instructional and professional teacher competence frameworks, the disciplinary climate concept has also found its way into the frameworks of school climate. In these frameworks, a good disciplinary climate is a subdimension of school safety and comprises conflict resolution; clarity, fairness, and consistency of rules; and the belief in school rules (M.-T. Wang & Degol, 2015). Bringing together the conceptualizations of disciplinary climate as part of instructional quality and school climate, Scherer and Nilsen (2017) found that a safe and orderly school environment is also characterized by good classroom management, which can result in better school achievement. Moreover, a good disciplinary climate forms the prerequisite for engaging in other instructional activities, such as cognitive activation and teacher support (Klieme, Pauli, & Reusser, 2009). In this sense, the disciplinary climate helps teachers create learning environments that support students’ learning.

A large body of research testifies to the consistently positive and significant relation between a good disciplinary climate and academic achievement across educational contexts, subject areas, and countries (e.g., Bellens et al., 2019; Berkowitz et al., 2017; Seidel & Shavelson, 2007; M. C. Wang, Haertel, & Walberg, 1993). However, this relation may vary in individual-level (student) data in which perceptions of disciplinary climate are assessed and classroom- or school-level data in which aggregated perceptions of disciplinary climate are evaluated with a certain reliability. For instance, Fauth, Decristan, Rieser, Klieme, and Büttner (2014) found a significant correlation between disciplinary climate and academic achievement for the classroom level but not the student level. In their study of eighth-graders, Blank and Shavit (2016) found significant relations at the student and classroom level but not at the school level. Considering this variation, the specification of the appropriate level of analysis is critical to interpreting the relation between disciplinary climate and academic achievement (Marsh et al., 2012).

1.2 Socioeconomic Status and Academic Achievement

SES represents the social standing or class of an individual or group and comprises measures of parental education, income, and occupation (APA, 2006; Willms & Tramonte, 2019). The concept serves as a proxy for possible inequalities with respect to students’ background, and it has been studied extensively in relation to educationally relevant outcome variables, especially academic achievement (Thomson, 2018). This perspective focuses on achievement as the output of education and quantifies the possible influence of unequal conditions (SES) on it (i.e., inequalities on average across social groups; Espinoza, 2007). Given the popularity of this perspective, a plethora of studies examining the SES–achievement relation exists across academic domains and school subjects. While reviewing this large body of research is beyond the scope of this chapter, the chapter brings to attention some knowns and unknowns.

Several research syntheses have agreed that a statistically significant and positive relation between SES and academic achievement exists across domains, SES measures, and measures of academic achievement (e.g., Broer, Bai, & Fonseca, 2019; Harwell, Maeda, Bishop, & Xie, 2016; Kim, Cho, & Kim, 2019; Scherer & Siddiq, 2019; Sirin, 2005; van Ewijk & Sleegers, 2010; White, 1982). Despite this consistent finding, the corresponding effect sizes ranged from small ($\overline{r}$ = 0.12) to moderate ($\overline{r}$ = 0.32) coefficients and varied across study, sample, and measurement characteristics (e.g., gender and grade-level composition in the sample, country of origin, types of achievement measures). Moreover, the statistical approaches most data analysts have taken to describe SES–achievement relations have been limited to correlational analyses of student-level data (Willms & Tramonte, 2019). This observation brings to light one key issue, that is, the appropriate level of analysis at which the SES–achievement relation is described. Clearly, students’ SES has a substantive meaning for individual students and is considered a powerful variable explaining achievement differences between students. At the same time, SES has a substantive meaning for classrooms and schools, representing the classroom or school SES composition (Thomson, 2018). Recognizing that SES and academic achievement can also be related at some level of clustering requires a multilevel approach to describing achievement gaps and composition effects (Marsh et al., 2009).

1.3 Three Hypotheses on the Interplay Between Disciplinary Climate, SES, and Academic Achievement

Bringing together the two lines of research describing the relation between SES and academic achievement and the relation between disciplinary climate and academic achievement, the core question this chapter assesses is how these three concepts play together. More specifically, while both lines of research have established significant links between the two pairs of concepts, the role of the disciplinary climate—as an aspect of both school climate and instructional quality—in academic achievement after controlling for SES, as well as the relation between academic achievement and SES, remains unclear.

Berkowitz et al. (2017) argued that the “scientific evidence establishing directional links and mechanisms between SES, school climate, and academic performance is inconclusive” (p. 425), especially due to the different perspectives educational researchers have taken to describe these links and mechanisms. Synthesizing these perspectives in 78 empirical studies, the authors identified three core hypotheses that describe the interplay between aspects of school climate, SES, and academic achievement; these are the compensation, mediation, and moderation hypotheses (see Fig. 8.1).

Fig. 8.1

Conceptual models framing of the relations among the three constructs. (Adopted from Willms & Tramonte, 2019)

The compensation hypothesis assumes that the disciplinary climate explains variation in academic achievement at the student and school levels above and beyond SES (Fig. 8.1a). It further assumes that the disciplinary climate contributes to “academic achievement beyond the expected outcomes based on SES background” (Berkowitz et al., 2017, p. 426). In this sense, support for this hypothesis could be interpreted as evidence for a compensating effect of disciplinary climate. Notably, this hypothesis does not make any assumptions on the link between SES and disciplinary climate—it only considers these two concepts as explanatory variables of academic achievement side-by-side, and therefore, it is commonly tested using contextual or single-level regression models. In their systematic review, Berkowitz et al. (2017) noticed that the compensation hypothesis is the dominating perspective researchers take to describe the interplay between SES, achievement, and climate variables. In the context of large-scale international assessments, indeed, many studies tested this hypothesis and obtained evidence that climate variables (represented as instructional quality or school climate) were significantly (and positively) related to academic achievement beyond SES at the student level and some level of clustering (e.g., Bellens et al., 2019; Ning et al., 2015; Rjosk et al., 2014; Shin, Lee, & Kim, 2009). This hypothesis takes the perspective of equality as it describes the relation between SES and educational outcomes—however, it only considers the additional variance explanation in educational outcomes through instructional variables (i.e., schooling) without a link between differences in SES and differences in disciplinary climate.

The mediation hypothesis assumes a mechanism underlying the relation between SES and academic achievement via disciplinary climate (Fig. 8.1b). Researchers testing this hypothesis argue that “a school’s SES influences its social climate, which in turn influences academic achievement” (Berkowitz et al., 2017, p. 426). In this sense, schools with a low average SES may struggle with establishing safe and orderly learning environments, and thus, be more likely to show low achievement (G. Chen & Weikart, 2008). Despite the causal claims behind this hypothesis, it is worth noting that the mediation mechanism is considered a school- or classroom-level mechanism rather than a student-level one (Liu et al., 2015). However, classroom or school climate variables are often assessed via student ratings, which are aggregated to the classroom or school level (Marsh et al., 2012); this allows researchers to test this hypothesis for individual students’ perceptions. In a slightly different context, Schmidt, Burroughs, Zoido, and Houang (2015) tested for student-level mediation and found support for significant indirect effects of individual SES on academic achievement via perceptions of opportunities to learn. In contrast to the moderation hypothesis, the mediation hypothesis adds the link between SES and disciplinary climate and an equity perspective to the compensation hypothesis by considering possible gaps in encountering or having access to a positive disciplinary climate. It also assumes a sequence of relations among variables, that is, SES → Disciplinary Climate → Achievement. Such a sequence entails that variation in disciplinary climate may be due to variation in SES, while variation in achievement may be due to variation in the disciplinary climate. Typically, multilevel mediation models are used to test this hypothesis (Preacher, Zyphur, & Zhang, 2010). This hypothesis takes the perspective of equity, as it describes the relation between SES and instructional variables (i.e., opportunities to experience instructional quality even with different needs resulting from varying socioeconomic background) and the perspective of equality, as it describes the SES–achievement relation. However, the SES–achievement relation is established only in the case of partial mediation and does not exist in the case of full mediation.

The moderation hypothesis assumes that the disciplinary climate may explain variation in the relation between students’ SES and their individual achievement across classrooms or schools (Fig. 8.1c). In other words, classrooms or schools of different disciplinary climate may show different SES–achievement relations (Berkowitz et al., 2017). In the case of negative moderation effects, a positive disciplinary climate is associated with smaller achievement gaps in classrooms or schools (Nilsen, Bloemeke, Yang Hansen, & Gustafsson, 2016). However, some empirical studies found positive moderation effects that pointed to a widening of the achievement gaps with better disciplinary climate (Gustafsson, Nilsen, & Hansen, 2018), while others could not identify any significant moderation (Bellens et al., 2019). Typically, researchers use cross-level interaction models to test the moderation hypothesis and address the extent to which differences in classroom or school conditions are associated with smaller achievement gaps (Jehangir, Glas, & van den Berg, 2015). Put differently, school conditions may facilitate the reduction of inequalities among students and/or improve their educational outputs irrespective of their background. A variation of this hypothesis includes classroom or school SES as another predictor of SES–achievement next to disciplinary climate (Fig. 8.1d). This variation allows researchers to examine the moderation effects of disciplinary climate above and beyond those of SES. Although the moderation effects are interpreted in a way that establishes disciplinary climate as the moderator, the empirical models testing these effects also allow for an alternative interpretation, in which SES is considered the moderator. Such an interpretation would entail that the relation between disciplinary climate and science achievement is smaller in high-SES schools than it is in low-SES schools. The moderation hypothesis takes the perspective of equality, describing the relation between SES and educational outcomes and considering possible moderation effects to be effects of schooling (Willms & Tramonte, 2019). In this sense, disciplinary climate may decrease possible inequalities in educational achievement across social groups (Espinoza, 2007).

The three hypotheses represent three lenses through which the interplay between disciplinary climate as an aspect of school climate and instructional quality, SES, and achievement can be examined.

1.4 The Present Study

This study focuses on the relations between disciplinary climate, socioeconomic status, and achievement in the context of science. The reasons for focusing on the context of science education are manifold: First, science is considered a core subject across many educational systems, including those of the Nordic countries, and it is a core domain of the existing large-scale assessments, such as PISA and TIMSS, which inform educational policy making (Kavli, 2018). Second, many educational systems struggle to provide equal opportunities for students to learn science; such inequalities may result in less frequent career choices in science, and they may ultimately pose a threat to national economic and technological competitiveness and equity (OECD, 2017a). Third, career choices in science are not determined only by students’ attitudes toward and motivation to learn science; rather, a remarkable body of research has shown that this aspiration is also determined by students’ home background, the distribution of capital, and parents’ social status (Archer et al., 2012). Fourth, many countries around the world are promoting science education to provide students with equal opportunities to learn the subject (Bianchini, 2017). Fifth, inequalities in science education and achievement may create inequalities in science capital and vice versa; such inequalities affect students’ participation in society as scientifically literate citizens (Archer, Dawson, DeWitt, Seakins, & Wong, 2015).

Utilizing the PISA 2015 data of the Nordic countries (Denmark, Finland, Iceland, Norway, and Sweden), the secondary analyses were aimed at examining the evidence for the three dominating hypotheses on the role of disciplinary climate: the compensation, mediation, and moderation hypotheses (see Fig. 8.1). In light of these three hypotheses, this chapter addresses the following three research questions (RQs):

RQ 1:

To what extent does disciplinary climate explain variations in science achievement above and beyond socioeconomic status?

RQ 2:

To what extent does disciplinary climate mediate the relation between socioeconomic status and science achievement?

RQ 3:

To what extent does the disciplinary climate explain between-school variation in the relation between socioeconomic status and science achievement?

Given that indicators of disciplinary climate are commonly assessed via students’, parents’, teachers’, or principals’ reports (M.-T. Wang & Degol, 2015), these assumptions may hold not only at the individual (within) level, where perceptions of the disciplinary climate are in the focus, but also at the aggregated (between) level, where shared perceptions about the school are in focus (Marsh et al., 2012). In other words, the three hypotheses may be tested for different levels of analysis—in PISA 2015, these levels refer to the student and the school level, with disciplinary climate assessed via student reports. Accounting for the multilevel nature of the data, this study considers several types of specificity via the following approaches: (a) This study compares the evidence for the three hypotheses across the five participating Nordic countries, taking a comparative perspective, and at the same time, allowing for country specificity; (b) as noted above, this study tests the three hypotheses for the student and the school level, accounting for level specificity; and (c) this study explores the role of disciplinary climate for the relation between SES and science achievement across the three core dimensions of SES (i.e., education, income, and occupation; APA, 2006), allowing for SES measurement specificity. The information about the extent to which the three hypotheses can or cannot be supported across these specific conditions adds to the evidence base on the interplay between disciplinary climate, socioeconomic status, and academic achievement. To summarize, the present study examines disciplinary climate in science lessons in terms of the following issues: (a) whether it explains variation in science achievement above and beyond SES, (b) whether it mediates the relation between SES and science achievement, and (c) whether it moderates the relation between SES and science achievement. In this respect, the relation between SES and disciplinary climate (i.e., students’ reported disciplinary climate in the schools they were placed in) was considered to be an indicator of equity and interpreted as the degree to which students were given opportunities to access a good disciplinary climate in science lessons. The relation between SES and science achievement was considered an indicator of (in-)equality that provides information about the degree to which SES differences in achievement exist (Espinoza, 2007; Willms & Tramonte, 2019).

Although the approach taken in this study was guided by three hypotheses in the context of equity and equality, the country comparisons were mainly exploratory, especially with respect to the evidence for or against the existence of a “Nordic model.” Despite the lack of a clear definition and a measurable framework of a Nordic model of education (Lundahl, 2016), the main goals of the Nordic school systems converge in that they strive for equity, participation, and welfare (Antikainen, 2006). However, these commonalities do not ensure that equal opportunities to learn, or in the context of this study, equal access to a good and positive disciplinary climate, exist across the Nordic countries. In fact, there is some evidence of substantial differences between them (OECD, 2017b; Sortkær & Reimer, 2018). Moreover, the existing international large-scale assessments suggest that the Nordic countries are far from scoring equally in the core domains of reading, science, and mathematics, and although relatively small, differences in measures of SES have arisen (OECD, 2016, 2019). Hence, exploring the differences and similarities in the information the three hypothesized models provide about the interplay of SES, science achievement, and disciplinary climate addresses whether evidence for a Nordic model exists in relation to the present models. For instance, possible differences in the contextual effects of schools’ disciplinary climate on students’ science achievement after controlling for SES may point to the fact that the possibilities to contribute to a better science achievement above and beyond the SES differences may not be equally exploited or provided across the Nordic countries. At the same time, such cross-country differences should not be overinterpreted as evidence against a Nordic model of education, especially because of the lack of a clear-cut framework that defines the dimensions and indicators of the model and because common efforts to create equity in the Nordic countries may not necessarily lead to the same results in education systems (Blossing, Imsen, & Moos, 2014; Lundahl, 2016). In this sense, the present study explores rather than hypothesizing on cross-country differences and similarities in the proposed models and does not argue that similarities have been caused by a “Nordic model.”

2 Data and Methodological Approaches

2.1 PISA 2015 Science Data of the Nordic Countries

The sample underlying the secondary analyses of the PISA 2015 data comprised the student samples of five Nordic countries, namely, Denmark, Finland, Iceland, Norway, and Sweden. Table 8.1 provides a brief summary of the corresponding sample sizes and the intraclass correlations (ICC₁) of the relevant variables. Each variable exhibited substantial between-school variation, and thus, allowed for decomposing their variances into the corresponding within and between parts (Snijders & Bosker, 2012). Notably, the smallest intraclass correlation for science achievement was apparent for the Icelandic data, while the Swedish data exhibited the largest coefficient. The disciplinary climate scale score varied the most between schools for the Norwegian data and the least for the Finnish data. Finally, between-school variation in the SES measures varied across the measures; nonetheless, consistently across the Nordic countries, the least variation occurred for the home possessions (HOMEPOS) measure.

Table 8.1 Description of the Nordic country samples included in the secondary data analyses

2.1.1 Science Achievement

In PISA 2015, the concept of scientific literacy comprised the three following core competencies: explaining phenomena scientifically, evaluating and designing scientific enquiry, and interpreting data and evidence scientifically (OECD, 2017a). Through a series of tasks requiring these competencies in the content domains labeled “Physical,” “Living,” and “Earth and Space,” students’ science achievement was measured and represented as a set of plausible values (OECD, 2017b). The secondary analyses included the plausible values PV1SCIE-PV10SCIE as indicators of the overall scientific literacy, yet not the plausible values specific to the three competencies or the content domains due to their high intercorrelations. Readers are kindly referred to the PISA 2015 Technical Report for more details about the psychometric properties and the design of the scientific literacy assessment (OECD, 2017b).

2.1.2 Socioeconomic Status

Students’ socioeconomic status was measured by several indicators in PISA 2015. These indicators were summarized in three subscale scores by means of item response theory modeling as follows: highest parental education (HISEI), parental education (PARED), and HOMEPOS. Performing principal component analysis, these three scores were then combined with composite SES indicators, namely, the Index of Economic, Social, and Cultural Status (ESCS). Given the psychometric issues associated with this composite SES score (Cronbach’s α values ranged between 0.53 and 0.65 for the Nordic countries; see OECD, 2017b), this chapter presents the results of the separate analyses for each of the three subscale scores. Moreover, due to the considerable heterogeneity of factor loadings within and between countries, SES was not represented as a latent variable measured by the three subscales scores to avoid biased estimates of structural parameters in structural equation models (Rhemtulla, van Bork, & Borsboom, 2019).

2.1.3 Disciplinary Climate in School Science Lessons

The disciplinary climate in school science lessons was assessed by students’ ratings of five statements on a four-point scale ranging from 0 (Never or hardly ever) to 3 (Every lesson) (ST097; see OECD, 2017b). Some of these statements addressed the same aspect of disciplinary climate (e.g., “Students don’t listen to what the teacher says” [Q01] and “The teacher has to wait a long time for students to quiet down” [Q03]), and a two-level confirmatory factor analysis suggested that residual covariances among two pairs of items existed (i.e., Q04 − Q05, Q01–Q03) beyond a within and a between latent variable representing disciplinary climate. To circumvent these redundancies and avoid construct-irrelevant multidimensionality, the three items—Q01, Q02, and Q04—served as manifest indicators. The correlation between the within- and between-level latent variables with the scale score DISCLISCI and the perfect correlations found provide some evidence for the validity of this approach. The within-level reliabilities ranged between ω_W = 0.79 and ω_W = 0.84, and the between-level reliabilities ranged between ω_B = 0.98 and ω_B = 0.99 across countries for the disciplinary climate scale comprising the three items.

2.2 Multilevel Structural Equation Modeling of the PISA 2015 Science Data

2.2.1 Analytic Setup

To test the models representing the three hypotheses (see Table 8.1), multilevel structural equation modeling (MSEM) described the measurement and structural models at the student (within) and school (between) levels in the statistical software package Mplus 8.2 (Muthén & Muthén, 1998-2017). The representation of the corresponding statistical models is provided in Fig. 8.2; a more detailed description within the MSEM framework can be found in the Supplementary Material A1 (see Figs. A1 and A2). Extending multilevel regression modeling, MSEM allows researchers to account not only for sampling error but also measurement error using latent variables (Marsh et al., 2009). The observed variables were decomposed into their latent within and between parts and specified the corresponding measurement and structural models to test the three hypotheses. Specifically, disciplinary climate was represented as a latent variable at both the student and school level measured by three observed indicators; science achievement and the SES measures were represented by one observed variable each. All models were estimated by means of robust maximum likelihood estimation, and possible missing values were handled through the full-information maximum likelihood procedure. Moreover, the student and school weights were employed to adjust for possible selection bias and differences in the sampling probabilities. Student weights were scaled to the cluster and school weights to the sample. For the models involving the science achievement scores (i.e., the set of 10 plausible values), the analyses were performed for each plausible analysis separately and combined the resultant model parameters using Rubin’s combination rules. The Mplus software facilitates this procedure via the TYPE = IMPUTATION option.

Fig. 8.2

Representation of the three hypotheses as multilevel structural equation models.

Note. ACH Science achievement, DIS Disciplinary climate perceptions, SES Socioeconomic status, B Between, W Within. Random slopes are indicated in orange. The path coefficient ${b}_3^B$ is only estimated for the testing of the moderation hypothesis II, yet not moderation hypothesis I

For the cross-level interaction models (moderation hypotheses I and II), the information criteria (AIC and BIC) were used to compare competing models; models exhibiting lower AIC and BIC values were preferred. To back these comparisons, likelihood-ratio tests were performed to examine the differences between different cross-level interaction models. For the contextual and mediation models, model fit was evaluated with the help of several fit indices, including the Satorra-Bentler corrected chi-square statistic (SB-χ²), the comparative fit index (CFI), the root mean square error of approximation (RMSEA), and the level-specific standardized root mean square residual (SRMR_W, SRMR_B), and the partial saturation approach was performed to identify possible sources of misfit (Ryu, 2014). The common guidelines for evaluating the goodness-of-fit (CFI ≥ 0.95, RMSEA ≤0.06, and SRMR ≤0.08) served as additional sources of information (Kline, 2015). All models were estimated as single-group two-level models first and multigroup two-level models second; the latter allowed for the country-specific reporting of the relevant model parameters.

2.2.2 Evaluating the Disciplinary Climate Measurement Model

Students’ perceptions of the disciplinary climate were represented as a latent variable at both the student and the school levels. To ensure the cross-level measurement invariance of these two latent variables and establish the same meaning of the respective constructs, factor loadings were constrained to being equal across levels (Stapleton, Yang, & Hancock, 2016). To support this constraint, multilevel confirmatory factor analysis models with and without these equality constraints were compared using fit indices and chi-square difference testing. After establishing that cross-level invariance held for data of each of the five Nordic countries, multilevel CFA models were extended to multigroup multilevel CFA models and tested for cross-country measurement invariance of the latent variables at both the student and school levels. This testing procedure was needed to establish that a sufficient degree of comparability across countries and levels was given to meaningfully compare the relations among variables. All model comparisons were based on the differences in CFI, RMSEA, SRMR-within, SRMR-between, and chi-square difference testing following the commonly applied guidelines for invariance testing (i.e., ΔCFI ≤ −0.010, ΔRMSEA ≤0.015, ΔSRMR ≤0.030; (Chen, 2007).

For the construct of the disciplinary climate, the results provided evidence that cross-country metric invariance at both the student and school levels and cross-level metric invariance held; the changes in the model fit statistics after adding invariance constraints did not deteriorate the model fit substantially. The final multigroup multilevel CFA model imposing these invariance constraints showed a very good fit to the data, SB- χ²(18) = 51.6, p < .001, CFI = 0.998, RMSEA = 0.019, SRMR_W = 0.014, SRMR_B = 0.014. Furthermore, the factor loadings of all three items were high across countries (λ_W = 0.70–0.85, λ_B = 0.91–1.00).

2.2.3 Evaluating the Structural Models

After examining the measurement models of disciplinary climate, the structural models were estimated. The models testing the compensation hypothesis were contextual models with latent-variable centered predictors of science achievement (ACH), and the contextual effect (cont_DIS) was represented as the difference between the between-level (${b}_1^B$) and within-level (${b}_1^W$) direct effects of disciplinary climate (DIS), ${\mathrm{cont}}_{DIS}={b}_1^B-{b}_1^W$. The standardized contextual effect with the corresponding effect size ES₂ were obtained (Marsh et al., 2009; see Supplementary Material S1). To test the mediation hypothesis, multilevel mediation models with indirect effects of the SES measures on science achievement via disciplinary climate at both levels were estimated. Given that all these variables were measured at the student level and aggregated to the school level, these mediation models can be classified as 1–1-1 multilevel mediation models (Preacher et al., 2010), with a contextual indirect effect represented as the difference between the between-level (ind_B) and within-level (ind_W) indirect effects, cont_ind = ind_B − ind_W(Nagengast & Marsh, 2012). The standardized squared indirect effect served as the corresponding effect size (Lachowicz, Preacher, & Kelley, 2018). Finally, the moderation hypotheses were tested with the help of cross-level interaction models (Aguinis, Gottfredson, & Culpepper, 2013).

3 Results

3.1 Compensation Hypothesis (RQ 1)

As noted above, the compensation hypothesis accounted for the level and SES measure specificity in the PISA 2015 data. Along these lines, the subsequent reporting contains the corresponding regression coefficients for the student and the school level and each of the three SES measures in Table 8.2. The regression coefficients describe the relation between disciplinary climate and science achievement after controlling for SES at the student level (${\mathrm{b}}_1^{\mathrm{W}}$) and the school level (${\mathrm{b}}_1^{\mathrm{B}}$). Next to the variance explanations, they served as the criteria used to determine whether the compensation hypothesis could be supported (see Fig. 8.2). A representation of the results is provided in Fig. 8.3, and a more detailed description including the model fit indices is given in Supplementary Material S2.

Table 8.2 Standardized coefficients of the student- and school-level regression models (Compensation hypothesis; see Fig. 8.2)

Fig. 8.3

Variance explanations of science achievement at the student and the school level by SES and disciplinary climate (DIS).

Note. The variance explanation of DIS is based on models in which both SES and DIS were included—these values indicate the additional contribution of DIS to the variance explanation by the SES measure

3.1.1 Compensation Hypothesis at the Student Level

Consistent across countries and SES measures, students’ perceptions of disciplinary climate predicted their science achievement above and beyond SES, with standardized regression coefficients ranging between ${\mathrm{b}}_1^{\mathrm{W}}$ = 0.037 and ${\mathrm{b}}_1^{\mathrm{W}}$ = 0.102 and overall variance explanations between ${\mathrm{R}}_{\mathrm{W}}^2$ = 1.5% and ${\mathrm{R}}_{\mathrm{W}}^2$ = 6.3%. These coefficients varied slightly between countries, and the Swedish data exhibited the smallest compensation effects. SES was a consistently strong predictor of individual science achievement, and the measure of disciplinary climate perceptions added only between ${\mathrm{R}}_{\mathrm{W}}^2$ = 0.1% and 1.0% to the variance explanation by SES (see Fig. 8.3).

3.1.2 Compensation Hypothesis at the School Level

The school-level regression coefficients of the disciplinary climate measure ranged between ${\mathrm{b}}_1^{\mathrm{B}}$ = 0.098 and ${\mathrm{b}}_1^{\mathrm{B}}$ = 0.543 across countries and SES measures. Notably, the Icelandic, Norwegian, and Swedish data showed the largest effects across all SES measures and supported the compensation hypothesis at the school level. Except for the HOMEPOS measure, the Danish data also provided evidence backing the compensation hypothesis; however, there was no support for the Finnish data. The overall variance explanations at the school level ranged between ${\mathrm{R}}_{\mathrm{B}}^2$ = 35.2% and ${\mathrm{R}}_{\mathrm{B}}^2$ = 74.8%. As for the student-level data, SES was a consistently strong predictor of school science achievement—the measure of disciplinary climate added between ${\mathrm{R}}_{\mathrm{B}}^2$ = 0.9% and ${\mathrm{R}}_{\mathrm{B}}^2$ = 23.1% to this variance explanation. The largest added values occurred for the Norwegian data (SES measures HISEI and PARED) and the Swedish data (SES measure HOMEPOS; see Fig. 8.3).

3.1.3 Contextual Direct Effects

The contextual effects—that is, the effects of school-level disciplinary climate on individual science achievement after controlling for school SES, individual SES, and perceptions of disciplinary climate—were statistically significant only for the Norwegian and the Swedish data. These effects were positive and ranged between ES₂ = 0.14 and ES₂ = 0.53. Notably, these effect sizes varied between the SES measures. Specifically, while they were of similar size for the SES measures HISEI and PARED for both Norway and Sweden, they differed to a larger extent between the countries for the HOMEPOS measure, with a larger effect for the Swedish data. Moreover, the effect was the largest among all effects for Sweden.

3.2 Mediation Hypothesis (RQ 2)

To test the mediation hypothesis, the indirect effects, along with the squared standardized indirect effects as effect sizes for both the student and the school level, were examined. Figure 8.4 shows the resultant direct and indirect effects for all SES measures, countries, and levels, and the Supplementary Material S2 contains all relevant model parameters.

Fig. 8.4

Direct, indirect, and total effects of the SES measures on science achievement via disciplinary climate.

Note. Standardized path coefficients are shown

3.2.1 Mediation Hypothesis at the Student Level

Across all analytic conditions, there was no evidence supporting that the indirect within-level effects were different from zero. All effects were small, and the corresponding effect sizes were zero. Overall, the mediation hypothesis could not be supported for the student-level data.

3.2.2 Mediation Hypothesis at the School Level

In contrast to the student level, the mediation models at the school level exhibited significant and positive indirect effects for the Swedish data across all SES measures (ind_B = 0.160–0.220), with the highest value for the HOMEPOS measure. The corresponding effect sizes ranged between ν = 0.026 and ν = 0.048, and these can be considered small (Lachowicz et al., 2018).

3.2.3 Contextual Indirect Effects

Only in the case of the Swedish data did a positive and statistically significant difference between the school- and the student-level indirect effects occur across all SES measures. Nevertheless, this contextual effect surfaced because the indirect effect did not exist in the student-level model, whereas it was present in the school-level model.

3.3 Moderation Hypotheses (RQ 3)

Concerning the first moderation hypothesis (see Fig. 8.2c), there was evidence for a positive moderation of the relation between SES and science achievement only for the Swedish data and only for the SES measures HISEI, ${c}_1^B$ = 1.031, SE = 0.453, p = .023, and HOMEPOS, ${c}_1^B$ = 31.663, SE = 10.403, p = .002. These moderation effects suggested an increase in the SES–achievement relation with a better disciplinary climate. However, given the large standard errors, these effects must be interpreted with caution. No further cross-level interaction effects in the other countries and across the other analytic conditions could be found.

Concerning the second moderation hypothesis (see Fig. 8.2c), there was no evidence for the role of disciplinary climate in science lessons as a moderator of the SES-achievement relation. After introducing school SES as a possible moderator, the moderating effects of disciplinary climate for the Swedish data disappeared (see Supplementary Material S2). In fact, there was evidence for a significant cross-level interaction effect of school SES under the following conditions: (a) HISEI: Iceland, Norway, and Sweden, ${b}_3^B$ = 0.039–0.056, ps < .05; (b) HOMEPOS: Finland, ${b}_3^B$ = −64.106, SE = 12.322, p = .004; and (c) PARED: Denmark, ${b}_3^B$ = 1.242, SE = 0.566, p = .028, and Iceland, ${b}_3^B$ = 2.985, SE = 1.329, p = .025. While the SES-achievement relation was stronger for higher values of HISEI or PARED in countries with significantly positive moderation effects, the relation was smaller for higher values of HOMEPOS in the Swedish data. Once again, the latter effect must be interpreted with caution due to the large standard error. Nevertheless, the moderation by disciplinary climate was not supported, and the moderation by SES differed across countries and SES measures.

3.4 Summary of the Main Findings

Table 8.3 visualizes the main findings; overall, the testing of the three hypotheses revealed the following results:

• Compensation hypothesis: Consistent evidence for the relation between disciplinary climate perceptions and science achievement after controlling for SES at the student level across all countries and measures of SES was found. At the same time, these relations varied between countries, with Finland and Norway showing the largest and Sweden the smallest effects. The variance explanations over and above SES were consistently small. Moreover, consistent evidence was found supporting the compensation hypothesis for the school level, except for the Finnish sample across all SES measures and the Danish sample for the home possession measure. The effects varied across SES measures even within countries; the Norwegian and Swedish data indicated consistently strong relations between disciplinary climate and science achievement and substantial variance explanations over and above SES at the school level. Contextual effects—that is, the effects of school-level disciplinary climate on individual science achievement across countries—existed only for the Norwegian and Swedish sample, with larger effect sizes for the latter.

• Mediation hypothesis: There was no evidence supporting the mediation hypothesis for the student level; only the Swedish data supported the existence of an indirect effect at the school level, which was consistent across all SES measures.

• Moderation hypothesis: The moderation hypothesis was supported only for the Swedish data and the SES measures representing HOMEPOS and HISEI status.

Table 8.3 Summary of the main findings

4 Discussion

4.1 The Three Hypotheses in the Context of Equity and Equality

As educational inequalities exist in academic achievement due to differences in students’ SES, and ultimately, the classroom and school SES composition, identifying possible classroom and school factors that may compensate, mediate, or moderate these inequalities is key to educational research and policy making (Cresswell, Schwantner, & Waters, 2015). In this sense, the three hypotheses proposed by Berkowitz et al. (2017) provide different lenses through which the role of such factors can be investigated. Using this framework, this study focused on disciplinary climate in science lessons as a school factor and obtained evidence for or against the three hypotheses.

Specifically, in all three hypotheses, a link between SES and science achievement was assumed, which represented inequalities in educational outcomes (Willms & Tramonte, 2019). This link existed across the five Nordic countries and across the two levels of analysis, and indeed, indicated the presence of outcome inequalities due to differences in SES between students within schools and between schools. The consistent and moderate association between SES and achievement is well in line with the existing body of research and testifies to the strong explanatory power of SES (e.g., Kim et al., 2019; Sirin, 2005; Thomson, 2018). While striving for reducing the SES–achievement relation is a key goal for educational effectiveness and school improvement (Scherer & Nilsen, 2019), explaining the possible mechanisms through which it operates is almost equally important (Berkowitz et al., 2017). In fact, knowledge about these mechanisms can provide insights into the roles of classroom or school factors from different perspectives—the mechanisms examined through the three hypotheses in this chapter were based on different assumptions about the role of the disciplinary climate, and ultimately, provided different interpretations.

The evidence supporting the compensation hypothesis suggests that a good disciplinary climate is indeed related to better science achievement after controlling for SES. In other words, disciplinary climate may compensate for educational inequalities due to SES. Notably, this finding was consistent across the five Nordic countries for both students and schools. At the individual (student) level, the compensation mechanism indicates that more positive perceptions of disciplinary climate in science lessons are associated with better science achievement after controlling for possible SES differences between students within a school. At the school level, the same interpretation holds for shared perceptions of disciplinary climate, school-average SES, and science achievement (Ning et al., 2015). One may argue that schools in the sample of Nordic countries succeed in achieving high due to establishing a good disciplinary climate in lessons, independent of their SES composition (e.g., Bellens et al., 2019).

The limited evidence for the mediation and moderation hypotheses for the PISA 2015 Nordic data may have several explanations and interpretations, which are as follows:

• Mediation only occurred for some countries at the school level but not the student level. Consequently, for the present data, this assumption represents school-level mechanisms implying that a good disciplinary climate is more likely to be found in high-SES schools, and ultimately, contributes to better school achievement (Liu et al., 2015). However, the student-level assumption that high-SES students are more likely to perceive the disciplinary climate more positively, and in turn, achieve better, still needs to be backed conceptually and empirically.

• For the Norwegian data, mediation could not be detected due to the missing link between-school SES and disciplinary climate across all three SES measures. The Finnish and Icelandic data showed the same pattern for the HOMEPOS and PARED measures. Consistently across all SES measures, the Finnish data further showed insignificant relations between disciplinary climate and achievement. The missing SES–climate link may be interpreted as an indicator of equal opportunities for students in schools to experience a good disciplinary climate (Willms & Tramonte, 2019). The missing climate–achievement link may be interpreted as a lack of school effects that could contribute to better achievement—for the Finnish PISA 2015 data, disciplinary climate was not a predictor of science achievement at the school level after controlling for both variables for school SES.

• Moderation by disciplinary climate is based on the assumption that a positive disciplinary climate may be associated with smaller achievement gaps (Ning et al., 2015). As there was no evidence supporting this assumption, the hopes associated with disciplinary climate as a possible factor reducing inequalities in educational outcomes could not be fulfilled for the present data. However, this observation is in line with previous studies that could not identify moderation effects (e.g., Bellens et al., 2019). Notably, tracing such effects with sufficient power depends on several factors, including the complexity of the cross-level interaction models and the decomposition of the moderator variable into its within and between parts (Aguinis et al., 2013). Possible methodological issues may prohibit the substantive interpretation of the effects.

Concerning whether a uniform “Nordic model” regarding the three hypotheses exists, the findings indicated cross-country differences not only in the sizes of the relations among SES, disciplinary climate, and science achievement but also in the conclusions following them. These differences emerged for the compensation hypothesis in the Danish and Finnish data and for the mediation and moderation hypotheses for the Norwegian and Swedish data, yet without consistent effects across SES measures. This observation brings forward the question of possible explanations for these differences. Although desirable, the present data do not provide opportunities to explore direct causal explanations, and any explanation at the level of educational systems (e.g., considering educational reforms and policy making) would need to be substantiated by external data sources and (quasi-)experimental research designs (Rutkowski & Delandshere, 2016). In this sense, researchers are encouraged to explore and investigate possible explanatory variables for the differences identified in the study; such variables could offer further insights into what may characterize a “Nordic model.”

4.2 Limitations and Future Directions

The secondary data analyses and possible inferences drawn from their results have at least two limitations worth noting: First, the disciplinary climate was assessed by student ratings as part of the PISA 2015 background questionnaire, and the corresponding items referred to the “disciplinary climate in school science lessons.” This reference to the school level rather than the classroom level hinders classroom-level inferences (Scherer, Nilsen, & Jansen, 2016). Instead, given the level of analysis, the interpretation of the construct is more in line with that of school climate rather than instructional quality.

Second, some methodological approaches taken in the secondary data analyses have not yet been fully developed. For instance, little is known about the importance of cross-level measurement invariance in cross-level interaction models with moderating school-level variables that are aggregated student-level variables (Jak, 2019), especially when detecting the cross-level interaction effect. Moreover, some relations in the analytic models may be curvilinear rather than linear (Teig, Scherer, & Nilsen, 2018). In this sense, methodological research on these issues will help readers fully understand the models that are used to describe the relations among the three constructs.

5 Conclusions and Implications

The secondary data analyses of the PISA 2015 data from five Nordic countries resulted in consistent and robust evidence supporting the compensation hypothesis, that is, the disciplinary climate’s contribution to science achievement above and beyond SES at both the student and school levels. At the same time, only limited evidence supporting the mediation hypothesis—with some exceptions for school-level data—and the moderation hypothesis surfaced. These observations point to the following conclusions: (a) although educational inequalities may exist, a good disciplinary climate is associated with better science achievement; and (b) inequalities in the opportunities to experience a good disciplinary climate (due to differences in SES) may not translate into inequalities in science achievement. Considering these conclusions, this study has several implications: From a substantive perspective, the three hypotheses may indeed represent educationally relevant lenses through which the role of disciplinary climate for SES, academic achievement, and the SES–achievement relation could be examined. This chapter has shown that these hypotheses can be converted into testable statistical models. From a methodological perspective, any study investigating the interplay between disciplinary climate, SES, and achievement should consider several levels of analysis and examine the meaning of the construct at these levels (e.g., student perceptions vs. shared perceptions of students within a school). In addition, the study highlighted the importance of measurement invariance to facilitate similar construct meaning across countries and levels.

This chapter further reveals some implications for the understanding of equity and equality in school contexts: First, the hope that disciplinary climate—a core school condition and indicator of instructional quality—can compensate efficiently and directly for possible achievement gaps in the domain of science could not be substantiated with the present data and selection of countries. This calls into question possible compensatory mechanisms and effects of the disciplinary climate as a malleable contextual variable. Second, the mechanisms describing the role of school conditions for addressing possible achievement gaps are far from clear cut; in fact, the PISA 2015 data did not provide clear support for any of them. This implies that the researchers’ theoretical perspectives on equity and equality will mainly determine the evaluation of the specific mechanism. Third, the three mechanisms tested in the secondary analyses shed light on different aspects of equity and equality; while the moderation hypothesis is based on the suggestion that equality in education can be increased by school conditions, the mediation hypothesis considers the dependencies between equality and equity via school conditions.

Concerning the elements describing a Nordic model, the study revealed some homogeneity in the findings across these countries—and some heterogeneity as well. Consistently, a compensation mechanism describing the interplay between equality and school conditions arose at different levels of analysis; however, the other mechanisms could hardly be traced. In this sense, achievement differences in science can partly be compensated for by a positive disciplinary school climate—the school condition studied in this chapter. Therefore, it seems that this compensation mechanism represents an element of the Nordic model. However, these findings do not imply a possible reduction of achievement gaps in science through a better disciplinary climate.

1 Electronic Supplementary Material (S)

Supplementary Material S1

(PDF 462 kb)

Supplementary Material S2

(XLSX 19722 kb)