Abstract
The purpose of this study was to investigate the effectiveness of the ORIGO Stepping Stones program in a large suburban school district in the Midwestern U.S.. The sample included 11 elementary schools that implemented the program during the 2013–2016 school years. Findings are presented from teacher surveys, classroom observations, and analysis of standardized student mathematics achievement scores on the Northwest Evaluation Association Measurement of Academic Progress (NWEA MAP). Using multilevel models, the program did not demonstrate a detectably larger effect on mathematics achievement than the comparison. Classroom observations and views of teachers that support program implementation were explored. In terms of fidelity of implementation, findings suggested that the program was delivered differently from how it was intended. With respect to researchers and policymakers, the approach used to evaluating mathematics program effectiveness is informative and could be used as part of larger accountability systems.
Similar content being viewed by others
References
Blanchard, M. R., LePrevost, C. E., Tolin, A. D., & Gutierrez, K. S. (2016). Investigating technology-enhanced teacher professional development in rural, high-poverty middle schools. Educational Researcher, 45(3), 207–220. doi:10.3102/0013189x16644602.
Bronfenbrenner, U. (1977). Toward an experimental ecology of human development. American Psychologist, 32(7), 513–531. doi:10.1037//0003-066x.32.7.513.
Cheung, A. C. K., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis. Educational Research Review, 9(88), 88–113. doi:10.1016/j.edurev.2013.01.001.
Common Core State Standards Initiative. (2010). Common core state standards for mathematics (CCSSM). Washington, DC: National Governors Association Center for Best Practices and the Council of Chief State School Officers. Retrieved from http://www.corestandards.org/wp-content/uploads/Math_Standards.pdf.
Corcoran, R. P. (2014). Classroom observation system. Baltimore, Maryland: School of Education, Johns Hopkins University.
Corcoran, R. P. (2016a). An English language and literacy acquisition validation randomized controlled trial: study protocol. International Journal of Educational Research, 80, 60–73. doi:10.1016/j.ijer.2016.07.003.
Corcoran, R. P. (2016b). Principals on the path to excellence: Longitudinal, multisite clusterrandomized controlled trials of the national institute for school leadership’s executive development program. International Journal of Educational Research, 79, 64–75. doi:10.1016/j.ijer.2016.05.001.
Corcoran, R. P. (2017). Preparing principals to improve student achievement. Child & Youth Care Forum. doi:10.1007/s10566-017-9399-9.
Corcoran, R. P., & O’Flaherty, J. (2016a). Personality development during teacher preparation. Frontiers in Psychology, 7(1677). doi:10.3389/fpsyg.2016.01677.
Corcoran, R. P., & O’Flaherty, J. (2016b). Examining the impact of prior academic achievement on moral reasoning development among college students: A growth curve analysis. Journal of Moral Education, 45(4), 433–448.
Corcoran, R. P., & O’Flaherty, J. (2017). Executive function during teacher preparation. Teaching and Teacher Education, 63, 168–175. doi:10.1016/j.tate.2016.12.023.
Corcoran, R. P., & Ross, S. M. (2014). Around the corner: A randomized controlled trial study of a technology-enhanced approach to early literacy. Cypriot Journal of Educational Sciences, 9, 316–329. Available from URL http://sproc.org/archives/index.php/cjes/article/view/9.4.6/pdf_360.
Corcoran, R. P., & Tormey, R. (2012). How emotionally intelligent are pre-service teachers? Teaching and Teacher Education, 28, 750–759. doi:10.1016/j.tate.2012.02.007.
Curry, C. (2003). Universal design accessibility for all learners. Educational Leadership, 61(2), 55–60. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.483.7152&rep=rep1&type=pdf.
Darling-Hammond, L. (2008). Teacher learning that supports student learning. In B. Z. Presseisen (Ed.), Teaching for intelligence (2nd ed., pp. 91–101). Thousand Oaks: Corwin Press.
Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82(1), 405–432. doi:10.1111/j.1467-8624.2010.01564.x.
Duschl, R. A., Schweingruber, H. A., & Shouse, A. W. (Eds.). (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: National Academies Press.
Dynarski, M., Agodini, R., Heaviside, S., Novak, T., Carey, N., & Campuzano, L. (2007). Effectiveness of reading and mathematics software products: Findings from the first student cohort (NCEE 2007–4005). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from https://ies.ed.gov/ncee/pdf/20074005.pdf.
Ertmer, P. A., Ottenbreit-Leftwich, A. T., Sadik, O., Sendurur, E., & Sendurur, P. (2012). Teacher beliefs and technology integration practices: A critical relationship. Computers & Education, 59(2), 423–435. doi:10.1016/j.compedu.2012.02.001.
Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S., … Ji, P. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6(3), 151–175. doi:10.1007/s11121-005-5553-y.
Fuson, K. C. (2003). Developing mathematical power in whole number operations. In J. Kilpatrick, W. G. Martin, & D. Schifter (Eds.), A research companion to principles and standards for school mathematics (pp. 68–94). Reston, Virginia: National Council of Teachers of Mathematics.
Fuson, K. C., Wearne, D. Hiebert, J. C., Murray, H. G., Human, P. G., Olivier, A. I., … Fennema, E. (1997). Children’s conceptual structures for multidigit numbers and methods of multidigit addition and subtraction. Journal for Research in Mathematics Education, 28(2), 130–162. doi:10.2307/749759.
Garet, M., Wayne, A., Stancavage, F., Taylor, J., Eaton, M., Walters, K., … Hurlburt, S. (2011). Middle school mathematics professional development impact study: Findings after the second year of implementation (NCEE 2011–4024). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from https://ies.ed.gov/ncee/pubs/20114024/pdf/20114024.pdf.
Gaspard, H., Dicke, A., Flunger, B., Brisson, B. M., Häfner, I., Nagengast, B., & Trautwein, U. (2015). Fostering adolescents’ value beliefs for mathematics with a relevance intervention in the classroom. Developmental Psychology, 51(9), 1226–1240. doi:10.1037/dev0000028.
Hasselbring, T. S., & Williams-Glaser, C. H. (2000). Use of computer technology to help students with special needs. The Future of Children, 10(2), 102–122. doi:10.2307/1602691.
Heck, R. H., Thomas, S. L., & Tabata, L. N. (2014). Multilevel and longitudinal modeling with IBM SPSS (2nd ed.). New York, NY: Routledge.
Hill, H. C., Blunk, M. L., Charalambous, C. Y., Lewis, J. M., Phelps, G. C., Sleep, L., & Ball, D. L. (2008). Mathematical knowledge for teaching and the mathematical quality of instruction: An exploratory study. Cognition and Instruction, 26(4), 430–511. doi:10.1080/07370000802177235.
Isaacs, A. C., & Carroll, W. M. (1999). Strategies for basic-facts instruction. Teaching Children Mathematics, 5(9), 508–515. Retrieved from https://eric.ed.gov/?id=EJ586737.
Jackson, C. K., & Bruegmann, E. (2009). Teaching students and teaching each other: The importance of peer learning for teachers (NBER working paper no. 15202). Cambridge, MA: The National Bureau of Economic Research. Retrieved from http://www.nber.org/papers/w15202.
Johns Hopkins University (2015). Evaluation of the Origo Stepping Stones Program: Second Study. Funded grant by Origo Education, Grant Number: 116279, R. P. Corcoran, Principal Investigator.
Kamil, M. L., Intrator, S. M., & Kim, H. S. (2000). The effects of other technologies on literacy and literacy learning. In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research (Vol. 3, pp. 771–788). Mahwah, NJ: Lawrence Erlbaum.
Kilpatrick, J., & Quinn, H. (2009). Science and mathematics education: Education policy white paper. Washington, DC: National Academy of Education.
Kilpatrick, J., Swafford, J., & Findell, B. (2001). Adding it up: Helping children learn mathematics. Washington, DC: National Academy Press.
Kim, E., & Corcoran, R. P. (2017). How engaged are pre-service teachers in the United States?. Teaching and Teacher Education, 66, 12–23.
Latham, A. S. (1997). Technology and LD students: What is best practice? Educational Leadership, 55(3), 88. Retrieved from http://www.ascd.org/publications/educational-leadership/nov97/vol55/num03/-Technology-and-LD-Students@-What-Is-Best-Practice%C2%A2.aspx.
Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054. Retrieved from https://pdfs.semanticscholar.org/977d/8f707ca1882e093c4ab9cb7ff0515cd944f5.pdf.
Mullis, I. V. S., Martin, M. O., Beaton, A. E., Gonzalez, E. J., Kelly, D. L., & Smith, T. A. (1997). Mathematics achievement in the primary school years: IEA’s third international mathematics and science study (TIMSS). Chestnut Hill, Massachusetts: Center for the Study of Testing, Evaluation and Educational Policy, Boston College. Retrieved from http://timss.bc.edu/timss1995i/TIMSSPDF/amtimss.pdf.
National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, Virginia: National Council of Teachers of Mathematics.
Northwest Evaluation Association. (2013). Measure of academic progress (MAP). Portland, Oregon: NWEA. Retrieved from https://www.nwea.org/content/uploads/2013/02/MAP-for-Primary-Grades-Overview.pdf.
Norusis, M. J. (2012). IBM SPSS Statistics 19 advanced statistical procedures companion. New York, NY: Pearson.
Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects. Educational Evaluation and Policy Analysis, 26(3), 237–257. doi:10.3102/01623737026003237.
Organization for Economic Co-operation and Development. (2016). PISA 2015 results (vol. 1): Excellence and equity in education. Paris, France: OECD Publishing.
Parker, L. L. (2005). Language development technologies for young English learners. Berkeley, California: University of California, Office of the President.
Pianta, R. C., Barnett, W. S., Burchinal, M., & Thornburg, K. R. (2009). The effects of preschool education: What we know, how public policy is or is not aligned with the evidence base, and what we need to know. Psychological Science in the Public Interest, 10(2), 49–88. doi:10.1177/1529100610381908.
Powell, D. R. (2006). Families and early childhood interventions. In K. A. Renninger & I. E. Sigel (Eds.), Handbook of child psychology (Vol. 4, 6th ed., pp. 548–591). Hoboken, NJ: Wiley.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, California: Sage.
Ryba, K., Selby, L., & Nolan, P. (1995). Computers empower students with special needs. Educational Leadership, 53(2), 82–84. Retrieved from http://www.ascd.org/publications/educational-leadership/oct95/vol53/num02/In-New-Zealand-~-Computers-Empower-Students-with-Special-Needs.aspx.
Salaberry, M. R. (2001). The use of technology for second-language learning and teaching: A retrospective. Modern Language Journal, 85(1), 30–56. Retrieved from http://salaberry.rice.edu/files/2013/10/Salaberry_2001_MLJ.pdf.
Sanders, W. L., Wright, S. P., & Horn, S. P. (1997). Teacher and classroom context effects on student achievement: Implications for teacher evaluation. Journal of Personnel Evaluation in Education, 11(1), 57–67 http://link.springer.com/article/10.1023%2FA%3A1007999204543.
Sealey, P., & Noyes, A. (2010). On the relevance of the mathematics curriculum to young people. The Curriculum Journal, 21(3), 239–253. doi:10.1080/09585176.2010.504573.
Slavin, R. E., & Karweit, N. L. (1985). Effects of whole class, ability grouped, and individualized instruction on mathematics achievement. American Educational Research Journal, 22(3), 351–367. doi:10.2307/1162968.
Slavin, R. E., & Lake, C. (2008). Effective programs in elementary mathematics: A best-evidence synthesis. Review of Educational Research, 78(3), 427–515. doi:10.3102/0034654308317473.
Slavin, R. E., Madden, N. A., & Leavey, M. (1984). Effects of team assisted individualization on the mathematics achievement of academically handicapped and nonhandicapped students. Journal of Educational Psychology, 76(5), 813–819. doi:10.1037//0022-0663.76.5.813.
Stipek, D., Salmon, J. M., Givvin, K. B., Kazemi, E., Saxe, G., & MacGyvers, V. L. (1998). The value (and convergence) of practices suggested by motivation research and promoted by mathematics education reformers. Journal for Research in Mathematics Education, 29(4), 465–488. doi:10.2307/749862.
Stronge, J. H., Ward, T. J., & Grant, L. W. (2011). What makes good teachers good? A cross-case analysis of the connection between teacher effectiveness and student achievement. Journal of Teacher Education, 62(4), 339–355. doi:10.1177/0022487111404241.
Thornton, C. (1990). Strategies for the basic facts. In J. N. Payne (Ed.), Mathematics for the young child (pp. 133–151). Reston, Virginia: National Council of Teachers of Mathematics.
Tschannen-Moran, M., & Hoy, A. W. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17(7), 783–805. doi:10.1016/S0742-051X(01)00036-1.
Van de Walle, J. A., & Watkins, K. B. (1993). Early development of number sense. In R. J. Jenson (Ed.), Research for the classroom: Early childhood mathematics (pp. 127–150). Reston, Virginia: National Council of Teachers of Mathematics.
Wei, R. C., Darling-Hammond, L., Andree, A., Richardson, N., & Ophanos, S. (2009). Professional learning in the learning profession: A status report on teacher development in the United States and abroad. Dallas, Texas: National Staff Development Council.
What Works Clearinghouse [WWC]. (2014). What works clearinghouse procedures and standards handbook version 3.0. Washington, DC: Institute of Education Sciences. Retrieved from https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_procedures_v3_0_standards_handbook.pdf.
Wu, H. (2011). Phoenix rising: Bringing the common core state mathematics standards to life. American Educator, 35(3), 3–13. Retrieved from http://www.aft.org/sites/default/files/periodicals/Wu.pdf.
Zhao, Y. (2003). Recent developments in technology and language learning: A literature review and meta-analysis. CALICO Journal, 21(1), 7–27. Retrieved from https://calico.org/html/article_279.pdf.
Acknowledgments
Thanks to the study participants and other personnel involved in the project including: M. Laurenzano, Johns Hopkins University; J. Reilly, Johns Hopkins University; S. M. Ross, Johns Hopkins University.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Funding
This study was funded by Origo Education (Grant Number: 116279; R. P. Corcoran, Principal Investigator).
Rights and permissions
About this article
Cite this article
Corcoran, R.P. Preparing Teachers’ to Raise Students’ Mathematics Learning. Int J of Sci and Math Educ 16, 1169–1185 (2018). https://doi.org/10.1007/s10763-017-9819-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10763-017-9819-1