A multilevel analysis of the effects of disciplinary climate strength on student reading performance

Abstract

Climate strength was first conceptualised in the organisational psychology literature as the within-group agreement on the perceptions of climate. In contrast to the deep study of climate level, climate strength has not been clarified by school climate research. The purpose of this cross-cultural study is to identify the main effect of disciplinary climate strength on student reading performance, and its moderating effect on the relationship between disciplinary climate level and student reading performance. A multilevel analysis was conducted on 2009 data from the Programme for International Student Assessment (PISA) from Shanghai-China, Japan, and the United States (US). The result showed a significantly positive relationship between climate strength and student reading performance in Shanghai-China and the US. Moreover, a moderating effect of climate strength was found in Shanghai-China and the US. The effects of climate strength were further examined in strong and weak conditions. School-level predictors of climate strength were also examined. The differences in disciplinary climate strengths and their effects on student reading performance suggested some culture differences in these countries/areas.

Appendices

Appendix 1: Measurement

Disciplinary climate

The index of disciplinary climate is measured by five items in the student questionnaire: (1) the students don’t listen to what the teacher says, (2) there is noise and disorder, (3) the teacher has to wait a long time for the students to quiet down, (4) the students cannot work well, and (5) the students don’t start working for a long time after the lesson begins. There are four response categories for these items: “strongly disagree”, “disagree”, “agree” and “strongly agree”. All items are reversed before IRT scaling.

School leadership

The index of school leadership is measured by 14 items in the school questionnaire: (1) I make sure that the professional development activities of teachers are in accordance with the teaching goals of the school; (2) I ensure that teachers work according to the school’s educational goals; (3) I observe instruction in classrooms; (4) I use student performance results to develop the school’s educational goals; (5) I give teachers suggestions as to how they can improve their teaching; (6) I monitor students’ work; (7) when a teacher has problems in his/her classroom, I take the initiative to discuss matters; (8) I inform teachers about possibilities for updating their knowledge and skills; (9) I check to see whether classroom activities are in keeping with our educational goals; (10) I take exam results into account in decisions regarding curriculum development; (11) I ensure that there is clarity concerning the responsibility for coordinating the curriculum; (12) when a teacher brings up a classroom problem, we solve the problem together; (13) I pay attention to disruptive behaviour in classrooms, and (14) I take over lessons from teachers who are unexpectedly absent. The response categories are “strongly disagree”, “disagree”, “agree” and “strongly agree”.

Teacher behaviours

The index of teacher behaviour is measured by seven items in the school questionnaire: (1) teachers’ low expectations of students, (2) poor student–teacher relations, (3) teachers not meeting individual students’ needs, (4) teacher absenteeism, (5) staff resistance to change, (6) teachers being too strict with students, and (7) students not being encouraged to achieve their full potential. The response categories are “strongly disagree”, “disagree”, “agree” and “strongly agree”. All items are reversed for scaling.

Enjoyment of reading

The index of enjoyment of reading is measured by 11 items in the student questionnaire: (1) I read only if I have to; (2) reading is one of my favourite hobbies; (3) I like talking about books with other people; (4) I find it hard to finish books; (5) I feel happy if I receive a book as a present; (6) for me, reading is a waste of time; (7) I enjoy going to a bookstore or a library; (8) I read only to get information that I need; (9) I cannot sit still and read for more than a few minutes; (10) I like to express my opinions about books I have read; and (11) I like to exchange books with my friends. The four response categories are “strongly disagree”, “disagree”, “agree” and “strongly agree”. All items negatively phrased are reverse-scored for IRT scaling.

Appendix 2: Modelling

Multilevel model

Model 0

$${\text{Level }}1{:}\,{\text{PVrea}}{{\text{d}}_{ij}}={\beta_{0j}}+{ \varepsilon_{ij}}$$

$${\text{Level }}2{:}\,{\beta_{0j}}={\gamma_{00}}+{\mu_{0j}}$$

Model 1

$${\text{Level }}1{:}\,{\text{PVrea}}{{\text{d}}_{ij}}={\beta_{0j}}+{\beta_{1j}}{\text{Gende}}{{\text{r}}_{ij}}+{\beta_{2j}}{\text{Grad}}{{\text{e}}_{ij}}+{\beta_{3j}}{\text{ESC}}{{\text{S}}_{ij}}+{\beta_{4j}}{\text{Joyrea}}{{\text{d}}_{ij}}+{\varepsilon_{ij}}$$

$$\begin{gathered} {\text{Level }}2{:}\,{\beta_{0j}}={\gamma_{00}}+{\gamma_{01}}{\text{Typ}}{{\text{e}}_{1j}}+{\gamma_{02}}{\text{Typ}}{{\text{e}}_{2j}}+{\gamma_{03}}{\text{School Siz}}{{\text{e}}_j}+{\gamma_{04}}{\text{ESCS}}\_{\text{mea}}{{\text{n}}_j}+{\gamma_{05}}{\text{Leadershi}}{{\text{p}}_j}+{\gamma_{06}}{\text{Propcer}}{{\text{t}}_j}+{\gamma_{07}}{\text{Strati}}{{\text{o}}_j}+{\gamma_{08}}{\text{Teacbeh}}{{\text{a}}_j}+{{\mu}_{0{\text{j}}}} \hfill \\ \beta _{{1j}} = \gamma _{{10}} \quad \beta _{{2j}} = \gamma _{{20}} \quad \beta _{{3j}} = \gamma _{{30}} + \mu _{{3j}} \quad \beta _{{4j}} = \gamma _{{40}} + \mu _{{4j}} \hfill \\ \end{gathered}$$

Model 2

$${\text{Level }}1{:}\,{\text{PVrea}}{{\text{d}}_{ij}}={\beta_{0j}}+{\beta_{1j}}{\text{Gende}}{{\text{r}}_{ij}}+{\beta_{2j}}{\text{Grad}}{{\text{e}}_{ij}}+{\beta_{3j}}{\text{ESC}}{{\text{S}}_{ij}}+{\beta_{4j}}{\text{Joyrea}}{{\text{d}}_{ij}}+{\varepsilon_{ij}}$$

$$\begin{gathered} {\text{Level }}2{:}\,{\beta_{0j}}={\gamma_{00}}+{\gamma_{01}}{\text{Typ}}{{\text{e}}_{1j}}+{\gamma_{02}}{\text{Typ}}{{\text{e}}_{2j}}+{\gamma_{03}}{\text{School Siz}}{{\text{e}}_j}+{\gamma_{04}}{\text{ESCS}}\_{\text{mea}}{{\text{n}}_j}+{\gamma_{05}}{\text{Leadershi}}{{\text{p}}_j}+{\gamma_{06}}{\text{Propcer}}{{\text{t}}_j}+{\gamma_{07}}{\text{Strati}}{{\text{o}}_j}+{\gamma_{08}}{\text{Teacbeh}}{{\text{a}}_j}+{\gamma_{09}}{\text{Climate Leve}}{{\text{l}}_j}+{\gamma_{10}}{\text{Climate Strengt}}{{\text{h}}_j}+{\mu _{0j}} \hfill \\ \beta _{{1j}} = \gamma _{{10}} \quad \beta _{{2j}} = \gamma _{{20}} \quad \beta _{{3j}} = \gamma _{{30}} + \mu _{{3j}} \quad \beta _{{4j}} = \gamma _{{40}} + \mu _{{4j}} \hfill \\ \end{gathered}$$

Model 3

$${\text{Level }}1{:}\,{\text{PVrea}}{{\text{d}}_{ij}}={\beta _{0j}}+{\beta_{1j}}{\text{Gende}}{{\text{r}}_{ij}}+{\beta_{2j}}{\text{Grad}}{{\text{e}}_{ij}}+{\beta_{3j}}{\text{ESC}}{{\text{S}}_{ij}}+{\beta_{4j}}{\text{Joyrea}}{{\text{d}}_{ij}}+{\varepsilon _{ij}}$$

$$\begin{gathered} {\text{Level }}2{:}\,{\beta_{0j}}={\gamma _{00}}+{\gamma_{01}}{\text{Typ}}{{\text{e}}_{1j}}+{\gamma_{02}}{\text{Typ}}{{\text{e}}_{2j}}+{\gamma_{03}}{\text{School Siz}}{{\text{e}}_j}+{\gamma_{04}}{\text{ESCS}}\_{\text{mea}}{{\text{n}}_j}+{\gamma_{05}}{\text{Leadershi}}{{\text{p}}_j}+{\gamma_{06}}{\text{Propcer}}{{\text{t}}_j}+{\gamma_{07}}{\text{Strati}}{{\text{o}}_j}+{\gamma_{08}}{\text{Teacbeh}}{{\text{a}}_j}+{\gamma_{09}}{\text{Climate Leve}}{{\text{l}}_j}+{\gamma_{10}}{\text{Climate Strengt}}{{\text{h}}_j}+{\gamma_{11}}{\text{Climate Level}} \times {\text{Climate Strengt}}{{\text{h}}_j}+{\mu _{0j}} \hfill \\ \beta _{{1j}} = \gamma _{{10}} \quad \beta _{{2j}} = \gamma _{{20}} \quad \beta _{{3j}} = \gamma _{{30}} + \mu _{{3j}} \quad \beta _{{4j}} = \gamma _{{40}} + \mu _{{4j}} \hfill \\ \end{gathered}$$

Model 4

$${\text{Level }}1{:}\,{\text{PVrea}}{{\text{d}}_{ij}}={\beta_{0j}}+{\beta_{1j}}{\text{Gende}}{{\text{r}}_{ij}}+{\beta_{2j}}{\text{Grad}}{{\text{e}}_{ij}}+{\beta_{3j}}{\text{ESC}}{{\text{S}}_{ij}}+{\beta_{4j}}{\text{Joyrea}}{{\text{d}}_{ij}}+{\varepsilon_{ij}}$$

$$\begin{gathered} {\text{Level }}2{:}\,{\beta_{0j}}={\gamma_{00}}+{\gamma_{01}}{\text{Typ}}{{\text{e}}_{1j}}+{\gamma_{02}}{\text{Typ}}{{\text{e}}_{2j}}+{\gamma_{03}}{\text{School Siz}}{{\text{e}}_j}+{\gamma_{04}}{\text{ESCS}}\_{\text{mea}}{{\text{n}}_j}+{\gamma_{05}}{\text{Leadershi}}{{\text{p}}_j}+{\gamma_{06}}{\text{Propcer}}{{\text{t}}_j}+{\gamma_{07}}{\text{Strati}}{{\text{o}}_j}+{\gamma_{08}}{\text{Teacbeh}}{{\text{a}}_j}+{\gamma_{09}}{\text{Climate Leve}}{{\text{l}}_j}+{\gamma_{10}}{\text{Climate Strengt}}{{\text{h}}_j}+{\gamma_{11}}{\text{Strength Dumm}}{{\text{y}}_j}+{\mu_{0j}} \hfill \\ \beta _{{1j}} = \gamma _{{10}} \quad \beta _{{2j}} = \gamma _{{20}} \quad \beta _{{3j}} = \gamma _{{30}} + \mu _{{3j}} \quad \beta _{{4j}} = \gamma _{{40}} + \mu _{{4j}} \hfill \\ \end{gathered}$$

where PVread _ij , Gender _ij , Grade _ij , ESCS _ij and Joyread _ij indicate the reading performance, gender, grade level, ESCS and enjoyment of reading for student i in school j; Type_1j and Type_2j are dummy variables of the school type for school j; School Size _j , ESCS_mean _j , Leadership _j , Propcert _j , Stratio _j , Teacbeha _j , Climate Level_j, Climate Strength _j , Climate Level × Climate Strength _j , and Strength Dummy_j are the school size, school ESCS, school leadership, proportion of certified teachers, student–teacher ratio, teacher behaviours, school disciplinary climate level, school disciplinary climate strength, the interaction term of disciplinary climate level and strength, and dummy variable for strong and weak conditions for school j; µ_0j, µ_3j, µ_4j and ε _ij are error terms; in Model 2, Model 3 and Model 4, γ₀₉ and γ₁₀ indicate the main effects of climate level and climate strength; in Model 3, γ₁₁ indicates the interaction effect between climate level and climate strength; in Model 4, γ₁₁ indicates the difference in student achievement between schools with strong climate conditions and schools with weak climate conditions.

Regression

$${\text{Climate Strengt}}{{\text{h}}_j}={\beta_0}+{\beta_1}{\text{Typ}}{{\text{e}}_{1j}}+{\beta_2}{\text{Typ}}{{\text{e}}_{2j}}+{\beta_3}{\text{School Siz}}{{\text{e}}_j}+{\beta_4}{\text{ESCS}}\_{\text{mea}}{{\text{n}}_j}+{\beta_5}{\text{Leadershi}}{{\text{p}}_j}+{\beta_6}{\text{Propcer}}{{\text{t}}_j}+{\beta_7}{\text{Strati}}{{\text{o}}_j}+{\beta_8}{\text{Teacbeh}}{{\text{a}}_j}+{\varepsilon_j}$$

where Climate Strength _j is the school disciplinary climate strength for school j; Type_1j, Type_2j are dummy variables of the school type for school j; School Size _j , ESCS_mean _j , Leadership _j , Propcert _j , Stratio _j and Teacbeha _j are the school size, school ESCS, school leadership, proportion of certified teachers, student–teacher ratio and teacher behaviours for school j; ε _j is the error term.