Abstract
This study presents results from a project to develop and pilot a new type of performance assessment of the skills used when teaching reading and writing. To provide context for this new approach, we begin by identifying the different types of knowledge that have historically been the focus of teacher assessment. Next we describe the newly developed performance assessment and present results from a pilot with 59 teacher candidates. The results suggest that the performance tasks can provide reliable evidence of the skills used to teach a range of reading and writing content across the K-6 grade span. We conclude by discussing the importance of future studies to further evaluate the validity argument for this type of performance assessment. We suggest potential uses such as studying teacher knowledge and learning, evaluating teacher education and professional development, and providing formative feedback as part of teacher preparation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The full list of 82 peer reviewed studies is available upon request.
The TOEFL IBT writing has an alternate form reliability of .81 (https://www.ets.org/s/toefl/pdf/toefl_ibt_research_s1v3.pdf) and the NAEP grade 12 writing tasks have intraclass correlations in the .69 to .82 range. https://nces.ed.gov/nationsreportcard/tdw/analysis/2007/initial_itemscorerescore_2007writg12.aspx.
Both the Praxis “Principals of Teaching and Learning Grades K-6” (https://www.ets.org/s/praxis/pdf/technical_manual.pdf) and Advanced Placement Tests (https://www.ets.org/research/policy_research_reports/publications/report/1996/hxtl) combine multiple choice and constructed response items to increase domain representation and improve test reliability.
References
Agathangelou, S. A., & Charalambous, C. Y. (2020). Is content knowledge pre-requisite of pedagogical content knowledge? An empirical investigation. Journal of Mathematics Teacher Education, 24(4), 421–458. https://doi.org/10.1007/s10857-020-09466-0
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.
Bennett, R. E. (2011). Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice, 18(1), 5–25. https://doi.org/10.1080/0969594X.2010.513678
Binks-Cantrell, E., Washburn, E. K., Joshi, R. M., & Hougen, M. (2012). Peter effect in the preparation of reading teachers. Scientific Studies of Reading, 19(6), 526–536. https://doi.org/10.1080/10888438.2011.601434
Bos, C. S., Mather, N., Friedman Narr, R., & Babur, N. (1999). Interactive, collaborative professional development in early literacy instruction: Supporting the balancing act. Learning Disabilities Research & Practice, 14(4), 215–226. https://doi.org/10.1207/sldrp1404_4
Bos, C., Mather, N., Dickson, S., Podhajski, B., & Chard, D. (2001). Perceptions and knowledge of preservice and inservice educators about early reading instruction. Annals of Dyslexia, 51, 97–120. https://doi.org/10.1007/s11881-001-0007-0
Briggs, S. R., & Cheek, J. M. (1986). The role of factor analysis in the development and evaluation of personality scales. Journal of Personality, 54(1), 106–148. https://doi.org/10.1111/j.14676494.1986.tb00391.x
Carlisle, J. F., Kelcey, B., Rowan, B., & Phelps, G. (2011). Teachers’ knowledge about early reading: Effects on students’ gains in reading achievement. Journal of Educational Effectiveness, 4(4), 289–331. https://doi.org/10.1080/19345747.2010.539297
Cash, A. H., Cabell, S. Q., Hamre, B. K., DeCoster, J., & Pianta, R. C. (2015). Relating prekindergarten teacher beliefs and knowledge to children’s language and literacy development. Teaching and Teacher Education, 48, 97–105. https://doi.org/10.1016/j.tate.2015.02.003
Charalambous, C. Y., Hill, H. C., Chin, M., & McGinn, D. (2019). Mathematical content knowledge and knowledge for teaching: Exploring their distinguishability and contribution to student learning. Journal of Mathematics Teacher Education, 23, 579–613. https://doi.org/10.1007/s10857-019-09443-2
Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309–319. https://doi.org/10.1037/1040-3590.7.3.309
Cunningham, A. E., Etter, K., Platas, L., Wheeler, S., & Campbell, K. (2015). Professional development in emergent literacy: A design experiment of teacher study groups. Early Childhood Research Quarterly, 31, 62–77. https://doi.org/10.1016/j.ecresq.2014.12.002
de Kramer, R. M., Masters, J., O’Dwyer, L. M., Dash, S., & Russell, M. (2012). Relationship of online teacher professional development to seventh-grade teachers’ and students’ knowledge and practices in English language arts. The Teacher Educator, 47(3), 236–259. https://doi.org/10.1080/08878730.2012.685795
Depaepe, F., Verschaffel, L., & Kelchtermans, G. (2013). Pedagogical content knowledge: A systematic review of the way in which the concept has pervaded mathematics educational research. Teaching and Teacher Education, 34, 12–25. https://doi.org/10.1016/j.tate.2013.03.001
Duguay, A., Kenyon, D., Haynes, E., August, D., & Yanosky, T. (2016). Measuring teachers’ knowledge of vocabulary development and instruction. Reading and Writing, 29(2), 321–347. https://doi.org/10.1007/s11145-015-9598-4
Durkin, D. (1964). Phonics test for teachers. Teachers College Press.
Early, D. M., Maxwell, K. L., Ponder, B. D., & Pan, Y. (2017). Improving teacher-child interactions: A randomized controlled trial of making the most of classroom interactions and my teaching partner professional development models. Early Childhood Research Quarterly, 38, 57–70. https://doi.org/10.1016/j.ecresq.2016.08.005
Educational Testing Service. (2018). The Praxis study companion for elementary education: Content knowledge for teaching (7811). https://www.ets.org/s/praxis/pdf/7811.pdf
Fitzharris, L., Jones, M. B., & Crawford, A. (2008). Teacher knowledge matters in supporting young readers. The Reading Teacher, 6(5), 384–394. https://doi.org/10.1598/RT.61.5.3
Gersten, R., Dimino, J., Jayanthi, M., Kim, J. S., & Santoro, L. E. (2010). Teacher study group: Impact of the professional development model on reading instruction and student outcomes in first grade classrooms. American Educational Research Journal, 47(3), 694–739. https://doi.org/10.3102/0002831209361208
Graham, M., Milanowski, A., & Miller, J. (2012). Measuring and promoting inter-rater agreement of teacher and principal performance ratings. Center for Educator Compensation Reform. Retrieved from http://cecr.ed.gov/pdfs/Inter_Rater.pdf.
Grossman, P., Compton, C., Igra, D., Rondfeld, M., Shahan, E., & Williamson, P. (2009). Teaching practice: A cross-professional perspective. Teachers College Record, 111(9), 2055–2100.
Howe, M., Grierson, S. T., & Richmond, M. G. (1997). A comparison of teachers’ knowledge and use of content reading strategies in the primary grades. Reading Research and Instruction, 36, 305–324. https://doi.org/10.1080/19388079709558246
Jordan, R., Bratsch-Hines, M., & Vernon-Feagans, L. (2018). Kindergarten and first grade teachers’ content and pedagogical content knowledge of reading and associations with teacher characteristics at rural low-wealth schools. Teaching and Teacher Education, 74, 190–204. https://doi.org/10.1016/j.tate.2018.05.002
Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (pp. 17–64). Praeger. https://doi.org/10.1177/0265532211417210
Kelcey, B. (2011). Assessing the effects of teachers’ reading knowledge on students’ achievement using multilevel propensity score stratification. Educational Evaluation and Policy Analysis, 33, 458–482. https://doi.org/10.3102/0162373711415262
Kucan, L., Hapgood, S., & Palincsar, A. (2011). Teachers’ specialized knowledge for supporting student comprehension in text-based discussions. The Elementary School Journal, 112(1), 61–82. https://doi.org/10.1086/660689
Lane, H. B., Hudson, R. F., Leite, W. L., Kosanovich, M. L., Strout, M. T., Fenty, N. S., & Wright, T. L. (2009). Teacher knowledge about reading fluency and students’ fluency growth in Reading First schools. Reading & Writing Quarterly, 25(1), 57–86. https://doi.org/10.1080/10573560802491232
Leader-Janssen, E., & Rankin-Erickson, J. (2013). Preservice teachers’ content knowledge and self-efficacy for teaching reading. Literacy Research and Instruction, 52, 204–229. https://doi.org/10.1080/19388071.2013.781253
Luecht, R. M. (2017). Data and scale analysis for credentialing examinations. In Davis-Becker, & Buckendahl (Eds.), Testing in the professions: Credentialing policies and practice (pp. 123–152). Routledge.
Martin-Raugh, M. P., Reese, C. M., Tannenbaum, R. J., Steinberg, J. H., & Xu, J. (2016). Investigating the relevance and importance of English language arts content knowledge areas for beginning elementary school teachers (Research Memorandum No. RM-16-08). Princeton, NJ: Educational Testing Service. https://files.eric.ed.gov/fulltext/ED570638.pdf
Masters, J., de Kramer, R. M., O’Dwyer, L. M., Dash, S., & Russell, M. (2010). The effects of online professional development on fourth-grade English language arts teachers’ knowledge and instructional practices. Journal of Educational Computing Research, 43(3), 355–375. https://doi.org/10.2190/EC.43.3.e
McCombes-Tolis, J., & Feinn, R. (2008). Comparing teachers’ literacy-related knowledge to their state’s standards for reading. Reading Psychology, 29(3), 236–265. https://doi.org/10.1080/02702710801982258
McCutchen, D., Abbott, R. D., Green, L. B., Beretvas, S. N., Cox, S., Potter, N. S., Quiroga, T., & Gray, A. L. (2002). Beginning literacy: Links among teacher knowledge, teacher practice, and student learning. Journal of Learning Disabilities, 35(1), 69–86. https://doi.org/10.1177/002221940203500106
McNinch, G. (1977). Teachers’ knowledge of word pronunciation skills. Southern Journal of Educational Research, 11(3), 99–110.
Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1(1), 3–62. https://doi.org/10.1207/S15366359MEA0101_02
Moats, L. C. (1994). The missing foundation in teacher education; Knowledge of the structure of spoken and written language. Annals of Dyslexia, 44(1), 81–102. https://doi.org/10.1007/BF02648156
Neuman, S. B., & Cunningham, L. (2009). The impact of professional development and coaching on early language and literacy instructional practices. American Educational Research Journal, 46(2), 532–566. https://doi.org/10.3102/0002831208328088
Park, Y., Kiely, M. T., Brownell, M. T., & Benedict, A. E. (2019). Relationships among special education teachers’ knowledge, instructional practice and students’ performance in reading fluency. Learning Disabilities Research & Practice, 34, 85–96. https://doi.org/10.1111/ldrp.12193
Paris, S. G., Lipson, M. Y., & Wixson, K. K. (1983). Becoming a strategic reader. Contemporary Educational Psychology, 8(3), 293–316. https://doi.org/10.1016/0361-476X(83)90018-8
Phelps, G. (2009). Just knowing how to read isn’t enough! What teachers know about the content of reading. Educational Assessment, Evaluation, and Accountability, 21(2), 137–154. https://doi.org/10.1007/s11092-009-9070-6
Phelps, G., Bridgeman, B., Yan, F., Steinberg, J., Weren, B., & Zhou, J. (2020a). Preliminary evidence on measurement characteristics for the Foundational Assessment of Competencies for Teaching (FACT) performance tasks (Research Report No. RR-20-27). Educational Testing Service. https://doi.org/10.1002/ets2.12310
Phelps, G., Gitomer, D. H., Iaconangelo, C. J., Etkina, E., Seeley, L., & Vokos, S. (2020b). Developing assessments of content knowledge for teaching using evidence-centered design. Educational Assessment, 25(2), 91–111. https://doi.org/10.1080/10627197.2020.1756256
Phelps, G., & Schilling, S. (2004). Developing measures of content knowledge for teaching reading. Elementary School Journal, 105(1), 31–48. https://doi.org/10.1086/428764
Phillips, B. M., Oliver, F., Tabulda, G., Wood, C., & Funari, C. (2020). Preschool teachers’ language and vocabulary knowledge: Development and predictive associations for a new measure. Dyslexia, 26(2), 153–172. https://doi.org/10.1002/dys.1644
Piasta, S. B., Connor, C. M., Fishman, B., & Morrison, F. J. (2009). Teachers’ knowledge of literacy concepts, classroom practices, and student reading growth. Scientific Studies of Reading, 13, 224–248. https://doi.org/10.1080/10888430902851364
Piasta, S. B., Justice, L. M., O’Connell, A. A., Mauck, S. A., Weber-Mayrer, M., Schachter, R. E., & Spear, C. F. (2017). Effectiveness of large-scale, state-sponsored language and literacy professional development on early childhood educator outcomes. Journal of Research on Educational Effectiveness, 10(2), 354–378. https://doi.org/10.1080/19345747.2016.1270378
Piasta, S., Park, S., Farley, K., Justice, L., & O’Connell, A. (2019). Early childhood educators’ knowledge about language and literacy: Associations with practice and children’s learning. Dyslexia, 26(2), 137–152. https://doi.org/10.1002/dys.1612
Rude, R. R. (1981). A knowledge test of reading for elementary school teaching. Journal of Educational Research, 74(6), 411–418. https://doi.org/10.1080/00220671.1981.10885341
Sadoski, M., Norton, D. E., Rodriguez, M., Nichols, W. D., & Gerla, J. P. (1998). Preservice and inservice reading teachers’ knowledge of literary concepts and literary analysis. Reading Psychology, 19(3), 267–286. https://doi.org/10.1080/0270271980190302
Schachter, R. E., Spear, C. F., Piasta, S. B., Justice, L. M., & Logan, J. (2016). Early childhood educators’ knowledge, beliefs, education, experiences, and children’s language- and literacy-learning opportunities: What is the connection? Early Childhood Research Quarterly, 36, 281–294. https://doi.org/10.1016/j.ecresq.2016.01.008
Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14. https://doi.org/10.3102/0013189x015002004
Schutz, K. M., Danielson, K. A., & Cohen, J. (2019). Approximations in English language arts: Scaffolding a shared teaching practice. Teaching and Teacher Education., 81, 100–111. https://doi.org/10.1016/j.tate.2019.01.004
Snow, C. E., Griffin, P., & Burns, S. M. (2005). Knowledge to support the teaching of reading: Preparing teachers for a changing world. Jossey-Bass.
Spear-Swerling, L., & Cheesman, E. (2012). Teachers’ knowledge base for implementing response-to-intervention models in reading. Reading & Writing: An Interdisciplinary Journal, 25(7), 1691–1723. https://doi.org/10.1007/s11145-011-9338-3
Spear-Swerling, L., & Zibulsky, J. (2014). Making time for literacy: Teacher knowledge and time allocation in instructional planning. Reading and Writing: An Interdisciplinary Journal, 27(8), 1353–1378. https://doi.org/10.1007/s11145-013-9491-y
Squires, D., Canney, G. F., & Trevisan, M. S. (2009). Minding the gate: Data-driven decisions about the literacy preparation of elementary teachers. Journal of Teacher Education, 60(2), 131–141. https://doi.org/10.1177/0022487108330552
Washburn, E. K., Joshi, R. M., & Binks Cantrell, E. (2011). Are preservice teachers prepared to teach struggling readers? Annals of Dyslexia, 61(1), 21–43. https://doi.org/10.1007/s11881-010-0040-y
Washington, V. M., & Miller-Jones, D. (1989). Teacher interactions with nonstandard English speakers during reading instruction. Contemporary Educational Psychology, 14(3), 280–312. https://doi.org/10.1016/0361-476X(89)90016-7
Wong-Fillmore, L., & Snow, C. E. (2002). What teachers need to know about language. In C. T. Adger, C. E. Snow, & D. Christian (Eds.), What teachers need to know about language (pp. 7–54). Delta Systems Co.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Appendices
Appendix: Performance tasks
Explain and mode how to add punctuation to sentences (RLA 11)
Explain How to read with appropriate stress, intonation and phrasing (RLA 12)
Video: (Sofia reads rapidly, but without appropriate attention to punctuation or expression.)
Explain how to use evidence from a poem to justify a theme (RLA 13)
Explain how to improve the organization of informational writing (RLA 14)
Model how to use prefixes and root words to determine word meaning (RLA 21)
Rights and permissions
About this article
Cite this article
Phelps, G., Bridgeman, B. From knowing to doing: assessing the skills used to teach reading and writing. Read Writ 35, 2023–2048 (2022). https://doi.org/10.1007/s11145-022-10278-2
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11145-022-10278-2