# Relating Performance on Written Assessments to Features of Mathematics Discussion

- 36 Downloads

## Abstract

Many researchers have illustrated the multi-faceted nature of classroom mathematics discussions, but few have demonstrated the effect of discussion on students’ assessment performance. We developed and employed a discussion observation instrument in 20 third and fourth grade classrooms in an economically disadvantaged, linguistically diverse school district and used hierarchical linear modeling (HLM) to determine whether between-class variation in word-problem-test scores can be explained by levels of class discussion. Results suggest overall class discussion scores, as well as two specific discussion features, variety of approaches and opportunities to speak, are significantly related to test performance. These results suggest classroom instruction including high-level math discussion may improve students’ performance on written measures of achievement.

## Keywords

Hierarchical linear modeling Mathematical discussion Mathematics achievement Mathematics education Observation instrument## Notes

### Acknowledgments

The research reported here was supported by the Math Alliance II, a professional development project funded by California Department of Education’s Improving Teacher Quality State Grants Program (ITQ) State Agency for Higher Education (formerly the California Postsecondary Education Commission) awarded to Rebecca Ambrose, Robert Bayley, and Michelle Anderson.

## Supplementary material

## References

- Abedi, J., & Lord, C. (2001). The language factor in mathematics tests.
*Applied Measurement in Education, 14*(3), 219–234.Google Scholar - Akkus, R., & Hand, B. (2010). Examining teacher’s struggles as they attempt to implement dialogical interaction as part of promoting mathematical reasoning within their classrooms.
*International Journal of Science and Mathematics Education, 9*, 975–998.CrossRefGoogle Scholar - Assor, A. (2012). Allowing choice and nurturing an inner compass: Educational practices supporting students’ need for autonomy. In
*Handbook of research on student engagement*(pp. 421-439). Springer, Boston, MA.CrossRefGoogle Scholar - Banes, L. C., Ambrose, R. C., Bayley, R., Restani, R. M., & Martin, H. A. (2018). Mathematical classroom discussion as an equitable practice: Effects on elementary English learners’ performance.
*Journal of Language, Identity & Education, 17*(6), 416–433.CrossRefGoogle Scholar - Baxter, J. A., Woodward, J., & Olson, D. (2001). Effects of reform-based mathematics instruction on low achievers in five third-grade classrooms.
*The Elementary School Journal*, 529-547.Google Scholar - Boston, M. (2012). Assessing instructional quality in mathematics.
*The Elementary School Journal, 113*(1), 76–104.CrossRefGoogle Scholar - Boston, M., Bostic, J., Lesseig, K., & Sherman, M. (2015). A comparison of mathematics classroom observation protocols.
*Mathematics Teacher Educator, 3*(2), 154–175.CrossRefGoogle Scholar - Cirillo, M. (2013).
*What does research say the benefits of discussion in mathematics class are*? (Research Brief No. 19). Reston, VA: NCTM.Google Scholar - Charles, R. I., Fennell, R., Caldwell, J., Cavanagh, M., Sammons, K., Schlielack, J., et al. (2009).
*enVision Math*. Glenview, IL: Pearson Scott Foresman.Google Scholar - Chi, M. T. (2000). Self-explaining expository texts: The dual processes of generating inferences and repairing mental models.
*Advances in instructional psychology, 5*,*161*–*238*.Google Scholar - Cobo, P., & Fortuny, J. (2000). Social interactions and cognitive effects in contexts of area-comparison problem solving.
*Educational Studies in Mathematics, 42*(2), 115–140.CrossRefGoogle Scholar - Cohen, J. (1992). A power primer.
*Psychological bulletin, 112*(1), 155–159.CrossRefGoogle Scholar - Crespo, S., & Harper, F. K. (in press). Learning to pose collaborative mathematics problems with secondary prospective teachers.
*International Journal of Educational Research.*https://doi.org/10.1016/j.ijer.2019.05.003. - Diamond, J. B. (2007). Where the rubber meets the road: Rethinking the connection between high-stakes testing policy and classroom instruction.
*Sociology of Education, 80*(4), 285–313.CrossRefGoogle Scholar - Durkin, K., Star, J. R., & Rittle-Johnson, B. (2017). Using comparison of multiple strategies in the mathematics classroom: lessons learned and next steps.
*ZDM Mathematics Education, 49*(4), 585–597.Google Scholar - Escamilla, K., Hopewell, S., Butvilofsky, S., Sparrow, W., Soltero-González, L., Ruiz-Figueroa, O., & Escamilla, M. (2014).
*Biliteracy from the start: Literacy squared in action*(pp. 25–26). Philadelphia, PA: Caslon Publishing.Google Scholar - Evans, S., & Swan, M. (2014). Developing students’ strategies for problem solving in mathematics: The role of pre-designed “Sample Student Work”.
*Educational Designer, 2*(7), 1–31.Google Scholar - Featherstone, H., Crespo, S., Jilk, L. M., Oslund, J., Parks, A., & Wood, M. (2011).
*Smarter together! Collaboration and equity in the elementary math classroom*. Reston, VA: National Council of Teachers of Mathematics.Google Scholar - Forman, E. A., McCormick, D., & Donato, R. (1998). Learning what counts as a mathematical explanation.
*Linguistics and Education, 9*(4), 313–339.CrossRefGoogle Scholar - Fraivillig, J. L., Murphy, L. A., & Fuson, K. C. (1999). Advancing children’s mathematical thinking in everyday mathematics classrooms.
*Journal for Research in Mathematics Education, 30*(2), 148–170.CrossRefGoogle Scholar - Gallimore, R., Hiebert, J. & Ermeling, B. (2014). Rich classroom discussion: One way to get rich learning.
*Teachers College Record.*http://www.tcrecord.org ID Number: 17714. - García, O. (2009). Emergent bilinguals and TESOL: What's in a name?.
*Tesol Quarterly, 43*(2), 322–326.CrossRefGoogle Scholar - Halpin, P. F., & Kieffer, M. J. (2015). Describing profiles of instructional practice: A new approach to analyzing classroom observation data.
*Educational Researcher, 44*(5), 263–277.CrossRefGoogle Scholar - Heinze, A., Marschick, F., & Lipowsky, F. (2009). Addition and subtraction of three-digit numbers: Adaptive strategy use and the influence of instruction in German third grade.
*ZDM Mathematics Education, 41*(5), 591–604.CrossRefGoogle Scholar - Hemmings, B., Grootenboer, P., & Kay, R. (2011). Predicting mathematics achievement: The influence of prior achievement and attitudes.
*International Journal of Science and Mathematics Education, 9*(3), 691–705.CrossRefGoogle Scholar - Hiebert, J., & Carpenter, T. P. (1992). Learning and teaching with understanding. In D. A. Grouws (Ed.),
*Handbook of research on mathematics teaching and learning*(pp. 65–97). New York: Macmillan.Google Scholar - Hiebert, J., & Grouws, D. A. (2007). The effects of classroom mathematics teaching on students’ learning. In F. Lester (Ed.),
*Second handbook of research on mathematics teaching and learning, 1*(pp. 371–404). Charlotte, NC: Information Age.Google Scholar - Hiebert, J., & Wearne, D. (1993). Instructional tasks, classroom discourse, and students’ learning in second-grade arithmetic.
*American Educational Research Journal, 30*(2), 393–425.CrossRefGoogle Scholar - Hox, J. J. (2010).
*Multilevel analysis: Techniques and applications*. New York: Routledge.CrossRefGoogle Scholar - Hufferd-Ackles, K., Fuson, K. C., & Sherin, M. G. (2004). Describing levels and components of a math-talk learning community.
*Journal for Research in Mathematics Education, 35*(2), 81–116.CrossRefGoogle Scholar - Jones, I., Swan, M., & Pollitt, A. (2015). Assessing mathematical problem solving using comparative judgment.
*International Journal of Science and Mathematics Education, 13*(1), 151–177.CrossRefGoogle Scholar - Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences.
*Educational Research Review, 2*, 130–144.CrossRefGoogle Scholar - Kazemi, E., & Hintz, A. (2013).
*Intentional talk: How to structure and lead productive mathematical discussions*. Stenhouse Publishers.Google Scholar - Kazemi, E., & Hintz, A. (2014).
*Intentional talk: How to structure and lead productive mathematical discussions*. Portland: Stenhouse Publishers.Google Scholar - Kazemi, E., & Stipek, D. (2001). Promoting conceptual thinking in four upper-elementary mathematics classrooms.
*Elementary School Journal, 102*(1), 59–80.CrossRefGoogle Scholar - Kline, P. (2000).
*Handbook of Psychological Testing*(2nd ed.). London: Routledge.Google Scholar - Knapp, M., Shields, P. M., & Turnbull, B. J. (1995). Academic challenge in high poverty classrooms.
*Phi Delta Kappan, 76*, 770–776.Google Scholar - Kosko, K., & Miyazaki, Y. (2012). The effect of student discussion frequency on fifth-grade students' mathematics achievement in U.S. schools.
*Journal of Experimental Education, 80*(2), 173–195. https://doi.org/10.1080/00220973.2011.566588.CrossRefGoogle Scholar - Lampert, M. (1990). When the problem is not the question and the solution is not the answer: Mathematical knowing and teaching.
*American educational research journal, 27*(1), 29-63.CrossRefGoogle Scholar - Long, J. S., & Freese, J. (2006).
*Regression models for categorical dependent variables using Stata*. Stata press.Google Scholar - Mathematics Learning Study Committee. (2001).
*Adding it up: Helping children learn mathematics*. Washington, DC: National Academies Press.Google Scholar - Matthews, P., & Rittle-Johnson, B. (2009). In pursuit of knowledge: Comparing self-explanations, concepts, and procedures as pedagogical tools.
*Journal of Experimental Child Psychology, 104*(1), 1–21.CrossRefGoogle Scholar - McEldoon, K. L., Durkin, K. L., & Rittle-Johnson, B. (2013). Is self-explanation worth the time? A comparison to additional practice.
*British Journal of Educational Psychology, 83*(4), 615–632.CrossRefGoogle Scholar - Mercer, N. (1996). The quality of talk in children's collaborative activity in the classroom.
*Learning and Instruction, 6*(4), 359–377.CrossRefGoogle Scholar - Mercer, N., & Sams, C. (2006). Teaching children how to use language to solve maths problems.
*Language and Education, 20*(6), 507–528.CrossRefGoogle Scholar - Merritt, E. G., Rimm-Kaufman, S. E., Berry, R. Q., Walkowiak, T. A., & Larsen, R. A. (2011).
*The contribution of mathematics instructional quality and class size to student achievement for third grade students from low income families. Paper presented at Fall Conference of Society for Research on Educational Effectiveness*.*D.C.*:*Washington*.Google Scholar - Moschkovich, J. (2013). Principles and guidelines for equitable mathematics teaching practices and materials for English language learners.
*Journal of Urban Mathematics Education, 6*(1), 45–57.Google Scholar - Murata, A. (2012). Diversity and high academic expectations without tracking: Inclusively responsive instruction.
*Journal of Learning Sciences, 22*(2), 312–335.CrossRefGoogle Scholar - National Council of Teachers of Mathematics. (1989).
*Curriculum and evaluation standards for school mathematics*. Reston, VA: NCTM.Google Scholar - National Council of Teachers of Mathematics. (2000).
*Principles and Standards for School Mathematics*. Reston, VA: NCTM.Google Scholar - National Council of Teachers of Mathematics. (2014).
*Principles to actions: Ensuring mathematical success for all*. Reston, VA: Author.Google Scholar - Norman, G. (2010). Likert scales, levels of measurement and the “laws” of statistics.
*Advances in health sciences education, 15*(5), 625–632.CrossRefGoogle Scholar - O'Connor, C., Michaels, S., Chapin, S., & Harbaugh, A. G. (2017). The silent and the vocal: Participation and learning in whole-class discussion.
*Learning and Instruction, 48*, 5–13.CrossRefGoogle Scholar - Otten, S., & Soria, V. M. (2014). Relationships between students’ learning and their participation during enactment of middle school algebra tasks.
*ZDM, 46*(5), 815–827.CrossRefGoogle Scholar - Pasta, D. J. (2009). Learning When to Be Discrete: Continuous vs. Categorical Predictors. In
*SAS Global Forum 2009*. San Francisco, CA: ICON Clinical Research.Google Scholar - Planas, N., & Gorgorió, N. (2004). Are different students expected to learn norms differently in the mathematics classroom?
*Mathematics Education Research Journal, 16*(1), 19–40.CrossRefGoogle Scholar - Raudenbush, S. W., & Bryk, A. S. (2002).
*Hierarchical linear models: Applications and data analysis methods*(2nd ed.). Thousand Oaks, CA: Sage.Google Scholar - Richland, L. E., Begolli, K. N., Simms, N., Frausel, R. R., & Lyons, E. A. (2017). Supporting mathematical discussions: the roles of comparison and cognitive load.
*Educational Psychology Review, 29*(1), 41–53.CrossRefGoogle Scholar - Rittle-Johnson, B., Saylor, M., & Swygert, K. E. (2008). Learning from explaining: Does it matter if mom is listening?
*Journal of Experimental Child Psychology, 100*(3), 215–224.CrossRefGoogle Scholar - Sawada, D., Piburn, Judson, E. M., Turley, J., Falconer, K., Benford, R., & Bloom, I. (2002). Measuring reform practices in science and mathematics classrooms: The reformed teaching observation protocol.
*School Science and Mathematics Journal, 102*, 245–253.CrossRefGoogle Scholar - Seidel, T., & Shavelson, R. J. (2007). Teaching effectiveness research in the past decade: The role of theory and research design in disentangling meta-analysis results.
*Review of Educational Research, 77*, 454–499. https://doi.org/10.3102/0034654307310317.CrossRefGoogle Scholar - Siegler, R. S. (1996).
*Emerging minds: The process of change in children's thinking*. New York: Oxford University Press.Google Scholar - Silver, E. A., Ghousseini, H., Gosen, D., Charalambou, C., & Font Strawhun, B. T. (2005). Moving from rhetoric to praxis: Issues faced by teachers in having students consider multiple solutions to problems in the mathematics classroom.
*Mathematics and Behavior, 24*, 287–301.CrossRefGoogle Scholar - Spillane, J. P., & Zeuli, J. S. (1999). Reform and teaching: Exploring patterns of practice in the context of national and state mathematics reforms.
*Educational Evaluation and Policy Analysis, 21*(1), 1–27.CrossRefGoogle Scholar - Stein, M. K., & Lane, S. (1996). Instructional tasks and the development of student capacity to think and reason: An analysis of the relationship between teaching and learning in a reform mathematics project.
*Educational Research and Evaluation, 2*(1), 50–80.CrossRefGoogle Scholar - Stein, M. K., Grover, B. W., & Henningsen, M. (1996). Building student capacity for mathematical thinking and reasoning: An analysis of mathematical tasks used in reform classrooms.
*American Educational Research Journal, 33*(2), 455–488.CrossRefGoogle Scholar - Stein, M. K., Correnti, R., Moore, D., Russell, J. L., & Kelly, K. (2017). Using theory and measurement to sharpen conceptualizations of mathematics teaching in the common core era.
*AERA Open, 3*(1), 2332858416680566.CrossRefGoogle Scholar - Stemler, S. E. (2004). A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability.
*Practical Assessment, Research & Evaluation, 9*(4), 1–19.Google Scholar - Sullivan, G. M., & Artino Jr., A. R. (2013). Analyzing and interpreting data from Likert-type scales.
*Journal of graduate medical education, 5*(4), 541–542.CrossRefGoogle Scholar - Walkowiak, T., Berry, R. Q., Meyer, J. P., Rimm-Kaufman, S. E., & Ottmar, E. R. (2014). Introducing an observational measure of standards-based mathematics teaching practices: Evidence of validity and score reliability.
*Educational Studies in Mathematics, 85*, 109128.CrossRefGoogle Scholar - Walshaw, M., & Anthony, G. (2008). The teacher’s role in classroom discourse: A review of recent research into mathematics classrooms.
*Review of Educational Research, 78*(3), 516–551.CrossRefGoogle Scholar - Webb, N. M., Franke, M. L., Ing, M., Wong, J., Fernandez, C. H., Shin, N., & Turrou, A. C. (2014). Engaging with others’ mathematical ideas: Interrelationships among student participation, teachers’ instructional practices, and learning.
*International Journal of Educational Research, 63*, 79–93.CrossRefGoogle Scholar