Data and learning that affords program improvement: a response to the U.S. accountability movement in teacher education

Abstract

The current U.S. education agenda places unprecedented attention on improving the quality of its roughly fifteen hundred teacher preparation programs. The effort places considerable weight on measuring program effectiveness and includes a push to hold programs accountable for their graduates’ effects on student performance. Concerns regarding the use of value-added methodologies for this purpose, combined with unsatisfactory measures historically in use (e.g., multiple-choice tests of teaching knowledge, surveys of program graduates), have led many states to adopt the use of a teaching performance measure (see, e.g., www.pacttpa.org and www.edtpa.aacte.org). States and programs have allocated significant resources to the use of multiple metrics in the quest for program accountability, but far less effort has been paid to the use of such metrics for program improvement. Given the rather limited research on how programs use these data to improve (see e.g., Peck and McDonald in New Educ 9(1):12–28, 2013), the following offers a case analysis of faculty learning as a function of data use for program improvement. In particular, the case presents an analysis of the types of data that mediated faculty learning within a program, and the organizational conditions that facilitated learning and program renewal.

Appendix

1.1 Glossary of terms

AACTE American Association of Colleges for Teacher Education (www.aacte.org). An organization that includes membership from over 800 US colleges and universities with teacher education programs. The organization describes its purpose as “advocacy and capacity building by promoting innovation and effective practices as critical to reforming educator preparation” (www.aacte.org about AACTE).

edTPA (www.edTPA.aacte.org): the performance-based assessment of teaching or TPA (see definition of TPA below) developed by Stanford University in partnership with the AACTE, is currently used in over 30 states for purposes of licensure and/or for program completion. The assessment requires candidates to submit a portfolio of teaching that is a “slice in time” of their practice (typically 3–5 h of instruction). The portfolio includes artifacts of teaching (such as lesson plans and K-12 student work), video of the teaching, and commentaries where candidates describe their rationale for planning and assessment, and analyze their teaching practice and their students’ learning. The portfolio is evaluated by trained assessors using 12–15 rubrics that have a five-point scale. Each level of the five-point rubric describes practice at different levels of sophistication. The portfolio requirements and rubrics are discipline and age specific (e.g., secondary History/Social Science, Elementary Mathematics, etc.). Teacher candidates must complete the PACT within the content area that they are seeking licensure in. In states where the edTPA is used for licensure, the state sets the passing score for the edTPA.

License (credential) and licensure within California and most states, a public school teacher must be licensed by a state board and/or a national accrediting organization (e.g., NCATE now CAEP www.caepnet.org). The policies for licensure are set by the states and require that teacher candidates complete an accredited teacher preparation program. In addition, many systems also require that candidates demonstrate that they are ready to teach through particular assessments as determined by the state (e.g., program designed assessments, a teacher performance assessment, and other state-approved measures of content and teaching). The term teaching “license” is synonymous with teaching “credential” in the State of California.

K-12 Kindergarten through grade 12. This is a term typically used to describe the compulsory US education system for students which spans from Kindergarten (age 5) through grade 12 (age 18) and includes 13 years of schooling.

NCATE and CAEP (www.caepnet.org) National Council on the Accreditation of Teacher Education (NCATE) which merged with a second national accrediting agency and is now called the Council for the Accreditation of Educator Preparation (CAEP). This is a national accrediting organization for the US teacher education programs. Any program in any state may seek CAEP accreditation, and some states require it. California requires its own state accreditation by the CTC, although many programs elect to also seek national accreditation by CAEP.

PACT the performance assessment for California teachers (www.pacttpa.org). This TPA (see definition below) is a performance-based assessment of teaching practice and was developed by a consortium of universities that included the University of California system campuses, Stanford, Mills College, and a number of campuses in the California State University system. It was approved by the CTC as one of the TPAs that program could adopt for use in the accreditation system to ensure that candidates were ready to teach. Candidates must pass the TPA in order to receive certification to teach in California. The assessment requires candidates to submit a portfolio of teaching that is a “slice in time” of their practice (typically 3–5 h of instruction). The portfolio includes artifacts of teaching (such as lesson plans and K-12 student work), video of the teaching, and commentaries where candidates describe their rationale for planning and assessment and analyze their teaching practice and their students’ learning. The portfolio requirements and rubrics are discipline and age specific (e.g., secondary History/Social Science, Elementary Mathematics, etc.). Teacher candidates must complete the PACT within the content area that they are seeking licensure in. The portfolio is evaluated by trained assessors using 13 rubrics that have a four-point scale. Each level of the four-point rubric describes practice at different levels of sophistication. As an example, the following is the first rubric for the planning task in elementary mathematics (for teaching mathematics in grades K-6):

Planning: establishing a balanced instructional focus
EM1: how do the plans support students’ development of conceptual understanding, computational/procedural fluency, and mathematical reasoning skills?
Level 1	Level 2	Level 3	Level 4
The standards, learning objectives, learning tasks, and assessments either have no central focus or a one-dimensional focus (e.g., all procedural or all conceptual)	The standards, learning objectives, learning tasks, and assessments have an overall focus that is primarily one-dimensional (e.g., procedural or conceptual)	Learning tasks or the set of assessment tasks focus on multiple dimensions of mathematics learning through clear connections among computations/procedures, concepts, and reasoning/problem solving strategies	Both learning tasks and the set of assessment tasks focus on multiple dimensions of mathematics learning through clear connections among computations/procedures, concepts, and reasoning/problem solving strategies
	The focus includes vague connections among computations/procedures, concepts, and reasoning/problem solving strategies	A progression of learning tasks and assessments is planned to build understanding of the central focus of the learning segment	A progression of learning tasks and assessments guides students to build deep understandings of the central focus of the learning segment

TPA teaching performance assessment. A general term that describes a performance-based measure of teaching practice.