# Preparing Teachers’ to Raise Students’ Mathematics Learning

## Abstract

The purpose of this study was to investigate the effectiveness of the ORIGO Stepping Stones program in a large suburban school district in the Midwestern U.S.. The sample included 11 elementary schools that implemented the program during the 2013–2016 school years. Findings are presented from teacher surveys, classroom observations, and analysis of standardized student mathematics achievement scores on the Northwest Evaluation Association Measurement of Academic Progress (NWEA MAP). Using multilevel models, the program did not demonstrate a detectably larger effect on mathematics achievement than the comparison. Classroom observations and views of teachers that support program implementation were explored. In terms of fidelity of implementation, findings suggested that the program was delivered differently from how it was intended. With respect to researchers and policymakers, the approach used to evaluating mathematics program effectiveness is informative and could be used as part of larger accountability systems.

## Keywords

Classroom observation Multilevel model Student achievement Teacher## Notes

### Acknowledgments

Thanks to the study participants and other personnel involved in the project including: M. Laurenzano, Johns Hopkins University; J. Reilly, Johns Hopkins University; S. M. Ross, Johns Hopkins University.

### Compliance with Ethical Standards

### Funding

This study was funded by Origo Education (Grant Number: 116279; R. P. Corcoran, Principal Investigator).

## References

- Blanchard, M. R., LePrevost, C. E., Tolin, A. D., & Gutierrez, K. S. (2016). Investigating technology-enhanced teacher professional development in rural, high-poverty middle schools.
*Educational Researcher, 45*(3), 207–220. doi: 10.3102/0013189x16644602.CrossRefGoogle Scholar - Bronfenbrenner, U. (1977). Toward an experimental ecology of human development.
*American Psychologist, 32*(7), 513–531. doi: 10.1037//0003-066x.32.7.513.CrossRefGoogle Scholar - Cheung, A. C. K., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis.
*Educational Research Review, 9*(88), 88–113. doi: 10.1016/j.edurev.2013.01.001.CrossRefGoogle Scholar - Common Core State Standards Initiative. (2010).
*Common core state standards for mathematics (CCSSM)*. Washington, DC: National Governors Association Center for Best Practices and the Council of Chief State School Officers. Retrieved from http://www.corestandards.org/wp-content/uploads/Math_Standards.pdf.Google Scholar - Corcoran, R. P. (2014).
*Classroom observation system*. Baltimore, Maryland: School of Education, Johns Hopkins University.Google Scholar - Corcoran, R. P. (2016a). An English language and literacy acquisition validation randomized controlled trial: study protocol.
*International Journal of Educational Research, 80,*60–73. doi: 10.1016/j.ijer.2016.07.003. - Corcoran, R. P. (2016b). Principals on the path to excellence: Longitudinal, multisite clusterrandomized controlled trials of the national institute for school leadership’s executive development program.
*International Journal of Educational Research, 79,*64–75. doi: 10.1016/j.ijer.2016.05.001. - Corcoran, R. P. (2017). Preparing principals to improve student achievement.
*Child & Youth Care Forum*. doi: 10.1007/s10566-017-9399-9. - Corcoran, R. P., & O’Flaherty, J. (2016a). Personality development during teacher preparation.
*Frontiers in Psychology, 7*(1677). doi: 10.3389/fpsyg.2016.01677. - Corcoran, R. P., & O’Flaherty, J. (2016b). Examining the impact of prior academic achievement on moral reasoning development among college students: A growth curve analysis.
*Journal of Moral Education, 45*(4), 433–448.Google Scholar - Corcoran, R. P., & O’Flaherty, J. (2017). Executive function during teacher preparation.
*Teaching and Teacher Education, 63*, 168–175. doi: 10.1016/j.tate.2016.12.023.CrossRefGoogle Scholar - Corcoran, R. P., & Ross, S. M. (2014). Around the corner: A randomized controlled trial study of a technology-enhanced approach to early literacy.
*Cypriot Journal of Educational Sciences, 9*, 316–329. Available from URL http://sproc.org/archives/index.php/cjes/article/view/9.4.6/pdf_360.Google Scholar - Corcoran, R. P., & Tormey, R. (2012). How emotionally intelligent are pre-service teachers?
*Teaching and Teacher Education, 28*, 750–759. doi: 10.1016/j.tate.2012.02.007.CrossRefGoogle Scholar - Curry, C. (2003). Universal design accessibility for all learners.
*Educational Leadership, 61*(2), 55–60. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.483.7152&rep=rep1&type=pdf.Google Scholar - Darling-Hammond, L. (2008). Teacher learning that supports student learning. In B. Z. Presseisen (Ed.),
*Teaching for intelligence*(2nd ed., pp. 91–101). Thousand Oaks: Corwin Press.Google Scholar - Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions.
*Child Development, 82*(1), 405–432. doi: 10.1111/j.1467-8624.2010.01564.x.CrossRefGoogle Scholar - Duschl, R. A., Schweingruber, H. A., & Shouse, A. W. (Eds.). (2007).
*Taking science to school: Learning and teaching science in grades K-8*. Washington, DC: National Academies Press.Google Scholar - Dynarski, M., Agodini, R., Heaviside, S., Novak, T., Carey, N., & Campuzano, L. (2007).
*Effectiveness of reading and mathematics software products: Findings from the first student cohort (NCEE 2007–4005)*. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from https://ies.ed.gov/ncee/pdf/20074005.pdf.Google Scholar - Ertmer, P. A., Ottenbreit-Leftwich, A. T., Sadik, O., Sendurur, E., & Sendurur, P. (2012). Teacher beliefs and technology integration practices: A critical relationship.
*Computers & Education, 59*(2), 423–435. doi: 10.1016/j.compedu.2012.02.001.CrossRefGoogle Scholar - Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S., … Ji, P. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination.
*Prevention Science*,*6*(3), 151–175. doi: 10.1007/s11121-005-5553-y. - Fuson, K. C. (2003). Developing mathematical power in whole number operations. In J. Kilpatrick, W. G. Martin, & D. Schifter (Eds.),
*A research companion to principles and standards for school mathematics*(pp. 68–94). Reston, Virginia: National Council of Teachers of Mathematics.Google Scholar - Fuson, K. C., Wearne, D. Hiebert, J. C., Murray, H. G., Human, P. G., Olivier, A. I., … Fennema, E. (1997). Children’s conceptual structures for multidigit numbers and methods of multidigit addition and subtraction.
*Journal for Research in Mathematics Education*,*28*(2), 130–162. doi: 10.2307/749759. - Garet, M., Wayne, A., Stancavage, F., Taylor, J., Eaton, M., Walters, K., … Hurlburt, S. (2011).
*Middle school mathematics professional development impact study: Findings after the second year of implementation (NCEE 2011–4024)*. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from https://ies.ed.gov/ncee/pubs/20114024/pdf/20114024.pdf. - Gaspard, H., Dicke, A., Flunger, B., Brisson, B. M., Häfner, I., Nagengast, B., & Trautwein, U. (2015). Fostering adolescents’ value beliefs for mathematics with a relevance intervention in the classroom.
*Developmental Psychology, 51*(9), 1226–1240. doi: 10.1037/dev0000028.CrossRefGoogle Scholar - Hasselbring, T. S., & Williams-Glaser, C. H. (2000). Use of computer technology to help students with special needs.
*The Future of Children, 10*(2), 102–122. doi: 10.2307/1602691.CrossRefGoogle Scholar - Heck, R. H., Thomas, S. L., & Tabata, L. N. (2014).
*Multilevel and longitudinal modeling with IBM SPSS*(2nd ed.). New York, NY: Routledge.Google Scholar - Hill, H. C., Blunk, M. L., Charalambous, C. Y., Lewis, J. M., Phelps, G. C., Sleep, L., & Ball, D. L. (2008). Mathematical knowledge for teaching and the mathematical quality of instruction: An exploratory study.
*Cognition and Instruction, 26*(4), 430–511. doi: 10.1080/07370000802177235.CrossRefGoogle Scholar - Isaacs, A. C., & Carroll, W. M. (1999). Strategies for basic-facts instruction.
*Teaching Children Mathematics, 5*(9), 508–515. Retrieved from https://eric.ed.gov/?id=EJ586737.Google Scholar - Jackson, C. K., & Bruegmann, E. (2009).
*Teaching students and teaching each other: The importance of peer learning for teachers (NBER working paper no. 15202)*. Cambridge, MA: The National Bureau of Economic Research. Retrieved from http://www.nber.org/papers/w15202. - Johns Hopkins University (2015). Evaluation of the Origo Stepping Stones Program: Second Study. Funded grant by Origo Education, Grant Number: 116279, R. P. Corcoran, Principal Investigator.Google Scholar
- Kamil, M. L., Intrator, S. M., & Kim, H. S. (2000). The effects of other technologies on literacy and literacy learning. In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, & R. Barr (Eds.),
*Handbook of reading research*(Vol. 3, pp. 771–788). Mahwah, NJ: Lawrence Erlbaum.Google Scholar - Kilpatrick, J., & Quinn, H. (2009).
*Science and mathematics education: Education policy white paper*. Washington, DC: National Academy of Education.Google Scholar - Kilpatrick, J., Swafford, J., & Findell, B. (2001).
*Adding it up: Helping children learn mathematics*. Washington, DC: National Academy Press.Google Scholar - Kim, E., & Corcoran, R. P. (2017). How engaged are pre-service teachers in the United States?.
*Teaching and Teacher Education, 66*, 12–23.Google Scholar - Latham, A. S. (1997). Technology and LD students: What is best practice?
*Educational Leadership, 55*(3), 88. Retrieved from http://www.ascd.org/publications/educational-leadership/nov97/vol55/num03/-Technology-and-LD-Students@-What-Is-Best-Practice%C2%A2.aspx.Google Scholar - Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge.
*Teachers College Record, 108*(6), 1017–1054. Retrieved from https://pdfs.semanticscholar.org/977d/8f707ca1882e093c4ab9cb7ff0515cd944f5.pdf.CrossRefGoogle Scholar - Mullis, I. V. S., Martin, M. O., Beaton, A. E., Gonzalez, E. J., Kelly, D. L., & Smith, T. A. (1997).
*Mathematics achievement in the primary school years: IEA’s third international mathematics and science study (TIMSS)*. Chestnut Hill, Massachusetts: Center for the Study of Testing, Evaluation and Educational Policy, Boston College. Retrieved from http://timss.bc.edu/timss1995i/TIMSSPDF/amtimss.pdf. - National Council of Teachers of Mathematics. (2000).
*Principles and standards for school mathematics*. Reston, Virginia: National Council of Teachers of Mathematics.Google Scholar - Northwest Evaluation Association. (2013).
*Measure of academic progress (MAP)*. Portland, Oregon: NWEA. Retrieved from https://www.nwea.org/content/uploads/2013/02/MAP-for-Primary-Grades-Overview.pdf. - Norusis, M. J. (2012).
*IBM SPSS Statistics 19 advanced statistical procedures companion*. New York, NY: Pearson.Google Scholar - Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects.
*Educational Evaluation and Policy Analysis, 26*(3), 237–257. doi: 10.3102/01623737026003237.CrossRefGoogle Scholar - Organization for Economic Co-operation and Development. (2016).
*PISA 2015 results (vol. 1): Excellence and equity in education*. Paris, France: OECD Publishing.Google Scholar - Parker, L. L. (2005).
*Language development technologies for young English learners*. Berkeley, California: University of California, Office of the President.Google Scholar - Pianta, R. C., Barnett, W. S., Burchinal, M., & Thornburg, K. R. (2009). The effects of preschool education: What we know, how public policy is or is not aligned with the evidence base, and what we need to know.
*Psychological Science in the Public Interest, 10*(2), 49–88. doi: 10.1177/1529100610381908.CrossRefGoogle Scholar - Powell, D. R. (2006). Families and early childhood interventions. In K. A. Renninger & I. E. Sigel (Eds.),
*Handbook of child psychology*(Vol. 4, 6th ed., pp. 548–591). Hoboken, NJ: Wiley.Google Scholar - Raudenbush, S. W., & Bryk, A. S. (2002).
*Hierarchical linear models: Applications and data analysis methods*(2nd ed.). Thousand Oaks, California: Sage.Google Scholar - Ryba, K., Selby, L., & Nolan, P. (1995). Computers empower students with special needs.
*Educational Leadership, 53*(2), 82–84. Retrieved from http://www.ascd.org/publications/educational-leadership/oct95/vol53/num02/In-New-Zealand-~-Computers-Empower-Students-with-Special-Needs.aspx.Google Scholar - Salaberry, M. R. (2001). The use of technology for second-language learning and teaching: A retrospective.
*Modern Language Journal, 85*(1), 30–56. Retrieved from http://salaberry.rice.edu/files/2013/10/Salaberry_2001_MLJ.pdf.CrossRefGoogle Scholar - Sanders, W. L., Wright, S. P., & Horn, S. P. (1997). Teacher and classroom context effects on student achievement: Implications for teacher evaluation.
*Journal of Personnel Evaluation in Education, 11*(1), 57–67 http://link.springer.com/article/10.1023%2FA%3A1007999204543.CrossRefGoogle Scholar - Sealey, P., & Noyes, A. (2010). On the relevance of the mathematics curriculum to young people.
*The Curriculum Journal, 21*(3), 239–253. doi: 10.1080/09585176.2010.504573.CrossRefGoogle Scholar - Slavin, R. E., & Karweit, N. L. (1985). Effects of whole class, ability grouped, and individualized instruction on mathematics achievement.
*American Educational Research Journal, 22*(3), 351–367. doi: 10.2307/1162968.CrossRefGoogle Scholar - Slavin, R. E., & Lake, C. (2008). Effective programs in elementary mathematics: A best-evidence synthesis.
*Review of Educational Research, 78*(3), 427–515. doi: 10.3102/0034654308317473.CrossRefGoogle Scholar - Slavin, R. E., Madden, N. A., & Leavey, M. (1984). Effects of team assisted individualization on the mathematics achievement of academically handicapped and nonhandicapped students.
*Journal of Educational Psychology, 76*(5), 813–819. doi: 10.1037//0022-0663.76.5.813.CrossRefGoogle Scholar - Stipek, D., Salmon, J. M., Givvin, K. B., Kazemi, E., Saxe, G., & MacGyvers, V. L. (1998). The value (and convergence) of practices suggested by motivation research and promoted by mathematics education reformers.
*Journal for Research in Mathematics Education, 29*(4), 465–488. doi: 10.2307/749862.CrossRefGoogle Scholar - Stronge, J. H., Ward, T. J., & Grant, L. W. (2011). What makes good teachers good? A cross-case analysis of the connection between teacher effectiveness and student achievement.
*Journal of Teacher Education, 62*(4), 339–355. doi: 10.1177/0022487111404241.CrossRefGoogle Scholar - Thornton, C. (1990). Strategies for the basic facts. In J. N. Payne (Ed.),
*Mathematics for the young child*(pp. 133–151). Reston, Virginia: National Council of Teachers of Mathematics.Google Scholar - Tschannen-Moran, M., & Hoy, A. W. (2001). Teacher efficacy: Capturing an elusive construct.
*Teaching and Teacher Education, 17*(7), 783–805. doi: 10.1016/S0742-051X(01)00036-1.CrossRefGoogle Scholar - Van de Walle, J. A., & Watkins, K. B. (1993). Early development of number sense. In R. J. Jenson (Ed.),
*Research for the classroom: Early childhood mathematics*(pp. 127–150). Reston, Virginia: National Council of Teachers of Mathematics.Google Scholar - Wei, R. C., Darling-Hammond, L., Andree, A., Richardson, N., & Ophanos, S. (2009).
*Professional learning in the learning profession: A status report on teacher development in the United States and abroad*. Dallas, Texas: National Staff Development Council.Google Scholar - What Works Clearinghouse [WWC]. (2014).
*What works clearinghouse procedures and standards handbook version 3.0*. Washington, DC: Institute of Education Sciences. Retrieved from https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_procedures_v3_0_standards_handbook.pdf.Google Scholar - Wu, H. (2011). Phoenix rising: Bringing the common core state mathematics standards to life.
*American Educator, 35*(3), 3–13. Retrieved from http://www.aft.org/sites/default/files/periodicals/Wu.pdf.Google Scholar - Zhao, Y. (2003). Recent developments in technology and language learning: A literature review and meta-analysis.
*CALICO Journal, 21*(1), 7–27. Retrieved from https://calico.org/html/article_279.pdf.CrossRefGoogle Scholar