A Review of the Literature on Teacher Effectiveness and Student Outcomes

Abstract

Researchers agree that teachers are one of the most important school-based resources in determining students’ future academic success and lifetime outcomes, yet have simultaneously had difficulties in defining what teacher characteristics make for an effective teacher. This chapter reviews the large body of literature on measures of teacher effectiveness, underscoring the diversity of methods by which the general construct of “teacher quality” has been explored, including experience, professional knowledge, and opportunity to learn. Each of these concepts comprises a number of different dimensions and methods of operationalizing. Single-country research (and particularly research from the United States) is distinguished from genuinely comparative work. Despite a voluminous research literature on the question of teacher quality, evidence for the impact of teacher characteristics (experience and professional knowledge) on student outcomes remains quite limited. There is a smaller, but more robust set of findings for the effect of teacher support on opportunity to learn. Five measures may be associated with higher student achievement: teacher experience (measured by years of teaching), teacher professional knowledge (measured by education and self-reported preparation to teach mathematics), and teacher provision of opportunity to learn (measured by time on mathematics and content coverage). These factors provide the basis for a comparative cross-country model.

Keywords

2.1 Defining Teacher Effectiveness

Researchers agree that teachers are one of the most important school-based resources in determining students’ future academic success and lifetime outcomes (Chetty et al. 2014; Rivkin et al. 2005; Rockoff 2004). As a consequence, there has been a strong emphasis on improving teacher effectiveness as a means to enhancing student learning. Goe (2007), among others, defined teacher effectiveness in terms of growth in student learning, typically measured by student standardized assessment results. Chetty et al. (2014) found that students taught by highly effective teachers, as defined by the student growth percentile (SGPs) and value-added measures (VAMs), were more likely to attend college, earn more, live in higher-income neighborhoods, save more money for retirement, and were less likely to have children during their teenage years. This potential of a highly effective teacher to significantly enhance the lives of their students makes it essential that researchers and policymakers properly understand the factors that contribute to a teacher’s effectiveness. However, as we will discuss in more detail later in this report, studies have found mixed results regarding the relationships between specific teacher characteristics and student achievement (Wayne and Youngs 2003). In this chapter, we explore these findings, focusing on the three main categories of teacher effectiveness identified and examined in the research literature: namely, teacher experience, teacher knowledge, and teacher behavior. Here we emphasize that much of the existing body of research is based on studies from the United States, and so the applicability of such national research to other contexts remains open to discussion.

2.2 Teacher Experience

Teacher experience refers to the number of years that a teacher has worked as a classroom teacher. Many studies show a positive relationship between teacher experiences and student achievement (Wayne and Youngs 2003). For example, using data from 4000 teachers in North Carolina, researchers found that teacher experience was positively related to student achievement in both reading and mathematics (Clotfelter et al. 2006). Rice (2003) found that the relationship between teacher experience and student achievement was most pronounced for students at the secondary level. Additional work in schools in the United States by Wiswall (2013), Papay and Kraft (2015), and Ladd and Sorenson (2017), and a Dutch twin study by Gerritsen et al. (2014), also indicated that teacher experience had a cumulative effect on student outcomes.

Meanwhile, other studies have failed to identify consistent and statistically significant associations between student achievement and teacher experience (Blomeke et al. 2016; Gustaffsson and Nilson 2016; Hanushek and Luque 2003; Luschei and Chudgar 2011; Wilson and Floden 2003). Some research from the United States has indicated that experience matters very much early on in a teacher’s career, but that, in later years, there were little to no additional gains (Boyd et al. 2006; Rivkin et al. 2005; Staiger and Rockoff 2010). In the first few years of a teacher’s career, accruing more years of experience seems to be more strongly related to student achievement (Rice 2003). Rockoff (2004) found that, when comparing teacher effectiveness (understood as value-added) to student test scores in reading and mathematics, teacher experience was positively related to student mathematics achievement; however, such positive relationships leveled off after teachers had gained two years of teaching experience. Drawing on data collected from teachers of grades four to eight between 2000 and 2008 within a large urban school district in the United States, Papay and Kraft (2015) confirmed previous research on the benefits experience can add to a novice teacher’s career. They found that student outcomes increased most rapidly during their teachers’ first few years of employment. They also found some further student gains due to additional years of teaching experience beyond the first five years. The research of Pil and Leana (2009) adds additional nuance; they found that acquiring teacher experience at the same grade level over a number of years, not just teacher experience in general (i.e. at multiple grades), was positively related to student achievement.

2.3 Teacher Professional Knowledge

A teacher’s professional knowledge refers to their subject-matter knowledge, curricular knowledge, and pedagogical knowledge (Collinson 1999). This professional knowledge is influenced by the undergraduate degrees earned by a teacher, the college attended, graduate studies undertaken, and opportunities to engage with on-the job training, commonly referred to as professional development (Collinson 1999; Rice 2003; Wayne and Youngs 2003). After undertaking in-depth quantitative analyses of the United States’ 1993–1994 Schools and Staffing Survey (SASS) and National Assessment of Educational Progress (NAEP) data sets, Darling-Hammond (2000) argued that measures of teacher preparation and certification were by far the strongest correlates of student achievement in reading and mathematics, after controlling for student poverty levels and language status.

As with experience, research on the impact of teacher advanced degrees, subject specializations, and certification has been inconclusive, with several studies (Aaronson et al. 2007; Blomeke et al. 2016; Hanushek and Luque 2003; Harris and Sass 2011; Luschei and Chudgar 2011) suggesting weak, inconsistent, or non-significant relationships with student achievement. However, several international studies comparing country means found that teacher degrees (Akiba et al. 2007; Gustaffsson and Nilson 2016; Montt 2011) were related to student outcomes, as did Woessman’s (2003) student-level study of multiple countries.

2.3.1 Undergraduate Education

In their meta-analysis of teacher effectiveness, Wayne and Youngs (2003) found three studies that showed some relationship between the quality of the undergraduate institution that a teacher attended and their future students’ success in standardized tests. In a thorough review of the research on teacher effectiveness attributes, Rice (2003) found that the selectivity of undergraduate institution and the teacher preparation program may be related to student achievement for students at the high school level and for high-poverty students.

In terms of teacher preparation programs, Boyd et al. (2009) found that overall these programs varied in their effectiveness. In their study of 31 teacher preparation programs designed to prepare teachers for the New York City School District, Boyd et al. (2009) drew from data based on document analyses, interviews, surveys of teacher preparation instructors, surveys of participants and graduates, and student value-added scores. They found that if a program was effective in preparing teachers to teach one subject, it tended to also have success in preparing teachers to teach other subjects as well. They also found that teacher preparation programs that focused on the practice of teaching and the classroom, and provided opportunities for teachers to study classroom practices, tended to prepare more effective teachers. Finally, they found that programs that included some sort of final project element (such as a personal research paper, or portfolio presentation) tended to prepare more effective teachers.

Beyond the institution a teacher attends, the coursework they choose to take within that program may also be related to their future students’ achievement. These associations vary by subject matter. A study by Rice (2003) indicated that, for teachers teaching at the secondary level, subject-specific coursework had a greater impact on their future students’ achievement. Similarly Goe (2007) found that, for mathematics, an increase in the amount of coursework undertaken by a trainee teacher was positively related to their future students’ achievement. By contrast, the meta-analysis completed by Wayne and Youngs (2003) found that, for history and English teachers, there was no evidence of a relationship between a teacher’s undergraduate coursework and their future students’ achievement in those subjects.

2.3.2 Graduate Education

In a review of 14 studies, Wilson and Floden (2003) were unable to identify consistent relationships between a teacher’s level of education and their students’ achievement. Similarly, in their review of data from 4000 teachers in North Carolina, Clotfelter et al. (2006) found that teachers who held a master’s degree were associated with lower student achievement. However, specifically in terms of mathematics instruction, teachers with higher degrees and who undertook more coursework during their education seem to be positively related to their students’ mathematics achievement (Goe 2007). Likewise, Harris and Sass (2011) found that there was a positive relationship between teachers who had obtained an advanced degree during their teaching career and their students’ achievement in middle school mathematics. They did not find any significant relationships between advanced degrees and student achievement in any other subject area. Further, using data from the United States’ Early Childhood Longitudinal Study (ECLS-K), Phillips (2010) found that subject-specific graduate degrees in elementary or early-childhood education were positively related to students’ reading achievement gains.

2.3.3 Certification Status

Another possible indicator of teacher effectiveness could be whether or not a teacher holds a teaching certificate. Much of this research has focused on the United States, which uses a variety of certification approaches, with lower grades usually having multi-subject general certifications and higher grades requiring certification in specific subjects. Wayne and Youngs (2003) found no clear relationship between US teachers’ certification status and their students’ achievement, with the exception of the subject area of mathematics, where students tended have higher test scores when their teachers had a standard mathematics certification. Rice (2003) also found that US teacher certification was related to high school mathematics achievement, and also found that there was some evidence of a relationship between certification status and student achievement in lower grades. Meanwhile, in their study of grade one students, Palardy and Rumberger (2008) also found evidence that students made greater gains in reading ability when taught by fully certified teachers.

In a longitudinal study using data from teachers teaching grades four and five and their students in the Houston School District in Texas, Darling-Hammond et al. (2005) found that those teachers who had completed training that resulted in a recognized teaching certificate were more effective that those who had no dedicated teaching qualifications. The study results suggested that teachers without recognized US certification or with non-standard certifications generally had negative effects on student achievement after controlling for student characteristics and prior achievement, as well as the teacher’s experience and degrees. The effects of teacher certification on student achievement were generally much stronger than the effects for teacher experience. Conversely, analyzing data from the ECLS-K, Phillips (2010) found that grade one students tended to have lower mathematics achievement gains when they had teachers with standard certification. In sum, the literature the influence of teacher certification remains deeply ambiguous.

2.3.4 Professional Development

Although work by Desimone et al. (2002, 2013) suggested that professional development may influence the quality of instruction, most researchers found that teachers’ professional development experiences showed only limited associations with their effectiveness, although middle- and high-school mathematics teachers who undertook more content-focused training may be the exception (Blomeke et al. 2016; Harris and Sass 2011). In their meta-analysis of the effects of professional development on student achievement, Blank and De Las Alas (2009) found that 16 studies reported significant and positive relationships between professional development and student achievement. For mathematics, the average effect size of studies using a pre-post assessment design was 0.21 standard deviations.

Analyzing the data from six data sets, two from the Beginning Teacher Preparation Survey conducted in Connecticut and Tennessee, and four from the United States National Center for Education Statistics’ National Assessment of Educational Progress (NAEP), Wallace (2009) used structural equation modeling to find that professional development had a very small, but occasionally statistically significant effect on student achievement. She found, for example, that for NAEP mathematics data from the year 2000, 1.2 additional hours of professional development per year were related to an increase in average student scores of 0.62 points, and for reading, an additional 1.1 h of professional development were related to an average increase in student scores of 0.24 points. Overall, Wallace (2009) identified professional development had moderate effects on teacher practice and some small effects on student achievement when mediated by teacher practice.

2.3.5 Teacher Content Knowledge

Of course, characteristics like experience and education may be imperfect proxies for teacher content knowledge; unfortunately, content knowledge is difficult to assess directly. However, there is a growing body of work suggesting that teacher content knowledge may associated with student learning. It should be noted that there is an important distinction between general content knowledge about a subject (CK) and pedagogical content knowledge (PCK) specifically related to teaching that subject, each of which may be independently related to student outcomes (Baumert et al. 2010).

Studies from the United States (see for example, Chingos and Peterson 2011; Clotfelter et al. 2006; Constantine et al. 2009; Hill et al. 2005; Shuls and Trivitt 2015) have found some evidence that higher teacher cognitive skills in mathematics are associated with higher student scores. Positive associations between teacher content knowledge and student outcomes were also found in studies based in Germany (Baumert et al. 2010) and Peru (Metzler and Woessman 2012), and in a comparative study using Programme for the International Assessment of Adult Competencies (PIAAC) data undertaken by Hanushek et al. (2018). These findings are not universal, however, other studies from the United States (Blazar 2015; Garet et al. 2016; Rockoff et al. 2011) failed to find a statistically significant association between teacher content knowledge and student learning.

The studies we have discussed all used some direct measure of teacher content knowledge. An alternative method of assessing mathematics teacher content knowledge is self-reported teacher preparation to teach mathematics topics. Both TIMSS and IEA’s Teacher Education and Development Study in Mathematics (TEDS-M, conducted in 2007–2008) have included many questions, asking teachers to report on their preparedness to teach particular topics. Although Luschei and Chudgar (2011) and Gustafsson and Nilson (2016) found that these items had a weak direct relationship to student achievement across countries, other studies have suggested that readiness is related to instructional quality (Blomeke et al. 2016), as well as content knowledge and content preparation (Schmidt et al. 2017), suggesting that instructional quality may have an indirect effect on student learning.

2.4 Teacher Behaviors and Opportunity to Learn

Although the impact of teacher characteristics (experience, education, and preparedness to teach) on student outcomes remains an open question, there is much a much more consistent relationship between student achievement and teacher behaviors (instructional time and instructional content), especially behaviors related instructional content. Analyzing TIMSS, Schmidt et al. (2001) found an association between classroom opportunity to learn (OTL), interpreted narrowly as student exposure to instructional content, and student achievement. In a later study using student-level PISA data, Schmidt et al. (2015) identified a robust relationship between OTL and mathematics literacy across 62 different educational systems. The importance of instructional content has been recognized by national policymakers, and has helped motivate standards-based reform in an effort to improve student achievement, such as the Common Core in the United States (Common Core Standards Initiative 2018). However, we found that there was little research on whether teacher instructional content that aligned with national standards had improved student learning; the only study that we were able to identify found that such alignment had only very weak associations with student mathematics scores (Polikoff and Porter 2014). Student-reported data indicates that instructional time (understood as classroom time on a particular subject) does seem to be related to mathematics achievement (Cattaneo et al. 2016; Jerrim et al. 2017; Lavy 2015; Rivkin and Schiman 2015; Woessman 2003).

2.5 Conclusion

This review of the literature simply brushes the surface of the exceptional body of work on the relationship between student achievement and teacher characteristics and behaviors. Whether analyzing US-based, international, or the (limited) number of comparative studies, the associations between easily measurable teacher characteristics, like experience and education, and student outcomes in mathematics, remains debatable. In contrast, there is more evidence to support the impact of teacher behaviors, such as instructional content and time on task, on student achievement. Our goal was to incorporate all these factors into a comparative model across countries, with the aim of determining what an international cross-national study like TIMSS could reveal about the influence of teachers on student outcomes in mathematics. The analysis that follows draws on the existing body of literature on teacher effectiveness, which identified key teacher factors that may be associated with higher student achievement: teacher experience, teacher professional knowledge (measured by education and self-reported preparation to teach mathematics), and teacher provision of opportunity to learn (time on mathematics and content coverage).