# A Lesson for the Common Core Standards Era from the NCTM Standards Era: The Importance of Considering School-Level Buy-in When Implementing and Evaluating Standards-Based Instructional Materials

## Abstract

As educators begin to implement new curriculum standards like the Common Core State Standards and the Next Generation Science Standards, data from earlier reform efforts can provide critical information about what factors contribute to or impede the ability of new instructional materials to improve student learning. We used data from a National Science Foundation (NSF) funded Local Systemic Change (LSC) initiative to investigate how school principal and teacher buy-in impacted the effectiveness of two middle school curricula designed to implement the NCTM Standards. We found that, compared to matched Comparison schools, Treatment schools with the highest buy-in saw substantial gains in mathematics achievement, whereas Treatment schools with the lowest buy-in saw substantial declines in mathematics achievement.

## Keywords

Longitudinal study Mathematics reform Teacher buy-in School support Student performance Standards-based curriculum Common Core Standards Curriculum implementation Teacher professional development## References

- Agodini, R., Harris, B., Thomas, M., Murphy, R., Gallagher, L., & Pendleton, A. (2010).
*Achievement effects of four elementary school math curricula: Findings for first and second graders.*Washington, DC: Department of Education. Retrieved Jan 12, 2013, from http://ies.ed.gov/ncee/pubs/20094052/index.asp - Banilower, E. R., Boyd, S. E., Pasley, J. K., & Weiss, I. R. (2006).
*Lessons from a decade of mathematics and science reform: A capstone report for the local systemic change through teacher enhancement initiative*. Chapel Hill, NC: Horizon Research, Inc.Google Scholar - Brown, A. L., & Campione, J. C. (1996). Psychological theory and the design of innovative learning environments: On procedures, principles, and systems. In R. Glaser (Ed.),
*Innovations in learning: New environments for education*(pp. 289–325). Mahwah, NJ: Erlbaum.Google Scholar - Brown, M. W., & Edelson, D. C. (2001, April).
*Teaching by design: Curriculum design as a lens on instructional practice.*Paper presented at the Annual Meeting of the American Educational Research Association, Seattle, WA.Google Scholar - Cai, J. (2003). What research tells us about teaching mathematics through problem solving. In F. Lester (Ed.),
*Research and issues in teaching mathematics through problem solving*(pp. 241–254). Reston, VA: National Council of Teachers of Mathematics.Google Scholar - Cai, J. (2010). Evaluation of mathematics education programs.
*International Encyclopedia of Education, 3*, 653–659.CrossRefGoogle Scholar - Cai, J., Nie, B., & Moyer, J. C. (2010). The teaching of equation solving: Approaches in
*Standards-based*and traditional curricula in the United States.*Pedagogies: An International Journal, 5*(3), 170–186.CrossRefGoogle Scholar - Cai, J., Wang, N., Moyer, J. C., Wang, C., & Nie, B. (2011). Longitudinal investigation of the curriculum effect: An analysis of student learning outcomes from the LieCal Project.
*International Journal of Educational Research, 50*(2), 117–136.CrossRefGoogle Scholar - Cho, J. (1998, April).
*Rethinking curriculum implementation: Paradigms, models, and teachers’ work.*Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.Google Scholar - Confrey, J., Castillo-Chavez, C., Grouws, D., Mahoney, C., Saari, D., Schmidt, W., et al. (2004).
*On evaluating curricular effectiveness: Judging the quality of K-12 mathematics evaluations*. Washington, DC: National Academies Press.Google Scholar - Council of Chief State School Officers and National Governors Association. (2010).
*Common core state standards for mathematics*. Washington, DC: Council of Chief State School Officers and National Governors Association.Google Scholar - Council of Chief State School Officers, Brookhill Foundation, & Texas Instruments. (2011).
*Common Core State Standards (CCSS) mathematics curriculum materials analysis project.*Washington, DC: Authors. Retrieved Jan 23, 2013, from https://www.k12.wa.us/CoreStandards/pubdocs/CCSSOMathAnalysisProj.pdf - Cross, C. T. (2004).
*Putting the pieces together: Lessons from comprehensive school reform research*. Washington, DC: National Clearinghouse for Comprehensive School Reform.Google Scholar - Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry.
*Educational Researcher, 32*(1), 5–8.CrossRefGoogle Scholar - Dobson, L. D., & Cook, T. J. (1980). Avoiding type III error in program evaluation: Results from a field experiment.
*Evaluation and Program Planning, 3*, 269–276.CrossRefGoogle Scholar - Flay, B., Biglan, A., Boruch, R., Castro, F., Gottfredson, D., Kellam, S., et al. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination.
*Prevention Science, 6*(3), 151–175.CrossRefGoogle Scholar - Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005). Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon model of parent management training.
*Behavior Therapy, 36*, 3–13.CrossRefGoogle Scholar - Fullan, M., & Pomfret, A. (1977). Research on curriculum and instruction implementation.
*Review of Educational Research, 47*, 335–397.CrossRefGoogle Scholar - Garet, M. S., Cronen, S., Eaton, M., Kurki, A., Ludwig, M., Jones, W., et al. (2008).
*The impact of two professional development interventions on early reading instruction and achievement (NCEE 2008-4030)*. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.Google Scholar - Glennan, T. K., Bodilly, S. J., Galegher, J. R., & Kerr, K. A. (2004).
*Expanding the reach of education reform: Perspectives from leaders in the scale-up of educational interventions.*Santa Monica, CA: Rand. Retrieved Jan 12, 2013, from http://www.rand.org/pubs/monographs/MG248.html - Goodlad, J. I. (1983). A study of schooling: Some implications for school improvement.
*Phi Delta Kappan, 64*(8), 552–558.Google Scholar - Hiebert, J. (1999). Relationships between research and the NCTM Standards.
*Journal for Research in Mathematics Education, 30*, 3–19.CrossRefGoogle Scholar - Hohmann, A. A., & Shear, M. K. (2002). Community-based intervention research: Coping with the “noise” of real life in study design.
*American Journal of Psychiatry, 159*, 201–207.CrossRefGoogle Scholar - Kennedy, M. M. (2004). Reform ideals and teachers’ practical intentions.
*Education Policy Analysis Archives, 12*(13). Retrieved Jan 2, 2013, from http://epaa.asu.edu/epaaa/v12n13/ - Kilpatrick, J. (2003). What works? In S. L. Senk & D. R. Thompson (Eds.),
*NSF funded school mathematics curricula: What they are? What do students learn?*(pp. 471–488). Mahwah, NJ: Erlbaum.Google Scholar - Krainer, K., & Peter-Koop, A. (2003). The role of the principal in mathematics teacher development. In A. Peter-Koop et al. (Eds.),
*Collaboration in teacher education*(pp. 169–190). Dordrecht: Kluwer Academic.CrossRefGoogle Scholar - Little, J. W. (1993). Teachers’ professional development in a climate of educational reform.
*Educational Evaluation and Policy Analysis, 15*, 129–151.CrossRefGoogle Scholar - National Council of Teachers of Mathematics. (1989).
*Curriculum and evaluation standards for school mathematics*. Reston, VA: National Council of Teachers of Mathematics.Google Scholar - National Council of Teachers of Mathematics. (2000).
*Principles and standards for school mathematics*. Reston, VA: The Council.Google Scholar - National Council of Teachers of Mathematics. (2009a).
*Curriculum focal points for prekindergarten through grade 8 mathematics: A quest for coherence*. Reston, VA: The Council.Google Scholar - National Council of Teachers of Mathematics. (2009b).
*Focus in high school mathematics: fostering reasoning and sense making for all students*. Reston, VA: The Council.Google Scholar - National Science Teachers Association. (2012).
*Recommendations on next generation science standards first public draft*. Arlington, VA: NSTA. Retrieved Feb 6, 2013, from http://www.nsta.org/about/standardsupdate/recommendations2.aspx - New Jersey Department of Education. (1996).
*New Jersey core curriculum content standards for mathematics*. Trenton, NJ: Author. Retrieved Aug 7, 2007, from http://www.edsolution.org/CustomizedProducts/data/standards-frameworks/standards/09mathintro.html - O’Donnell, C. L. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in KI-12 curriculum intervention research.
*Review of Educational Research, 78*(1), 33–84.CrossRefGoogle Scholar - O’Donnell, C. L. & Lynch, S. J. (2008, March).
*Fidelity of implementation to instructional strategies as a moderator of science curriculum unit effectiveness*. Paper presented at the annual meeting of the American Educational Research Association, New York. Retrieved Jan 12, 2013, from http://www.gwu.edu/~scale-up/documents/AERA%20O%27Donnell%20Lynch%202008%20-%20Fidelity%20of%20Implementation%20as%20a%20Moderator.pdf - Pennsylvania Department of Education. (1999).
*Academic standards for mathematics.*Harrisburg. PA: Author. Retrieved Aug 7, 2007, from http://www.pde.state.pa.us/k12/lib/k12/MathStan.doc - Remillard, J. T. (2005). Examining key concepts in research on teachers’ use of mathematics curricula.
*Review of Educational Research, 75*(21), 1–246.Google Scholar - Riordan, J. E., & Noyce, P. E. (2001). The impact of two standards-based mathematics curricula on student achievement in Massachusetts.
*Journal for Research in Mathematics Education, 32*(4), 368–398.CrossRefGoogle Scholar - Romberg, T. A., Folgert, L., & Shafer, M. C. (2005).
*Differences in student performances for three treatment groups.*(Mathematics in context longitudinal/cross-sectional study monograph 7). Madison, WI: University of Wisconsin, Wisconsin Center for Education Research. Retrieved May 28, 2014, from http://micimpact.wceruw.org/working_papers/Monograph%207%20Final.pdf - Rubin, D. (2001). Using propensity scores to help design observational studies: Application to the tobacco litigation.
*Health Services & Outcomes Research Methodology, 2*, 169–188.CrossRefGoogle Scholar - Schwartzbeck, T. D. (2002).
*Choosing a model and types of models: How to find what works for your school*. Washington, DC: National Clearinghouse for Comprehensive School Reform.Google Scholar - Slavin, R. E. (2002). Evidence-based educational policies: Transforming educational practice and research.
*Educational Researcher, 31*(7), 15–21.CrossRefGoogle Scholar - Slavin, R. E., Lake, C. & Groff, C. (2008).
*Effective programs in middle and high school mathematics: A best-evidence synthesis*. Baltimore, MD: Johns Hopkins University Center for Data Driven Reform in Education (CDDRE) Best Evidence Encyclopedia. Retrieved May 29, 2013, from http://www.bestevidence.org/word/mhs_math_Sep_8_2008.pdf - Stuart, E. A. (2007). Estimating causal effects using school-level data sets.
*Educational Researcher, 36*, 187–198.CrossRefGoogle Scholar - Tarr, J. E., Reys, R., Reys, B., Chávez, Ó., Shih, J., & Osterlind, S. (2008). The impact of middle school mathematics curricula on student achievement and the classroom learning environment.
*Journal for Research in Mathematics Education, 39*(3), 247–280.Google Scholar - Turnbull, B. (2002). Teacher participation and buy-in: Implications for school reform initiatives.
*Learning Environments Research, 5*(3), 235–252.CrossRefGoogle Scholar - U.S. Department of Education. (2007a).
*What works clearinghouse intervention report: connected mathematics project.*Washington, DC: Author. Retrieved Nov 1, 2007, from http://ies.ed.gov/ncee/wwc/pdf/WWC_CMP_040907.pdf - U.S. Department of Education. (2007b).
*Mathematics and science specific 84.305A RFA.*Washington, DC: Author. Retrieved Dec 2, 2007, from http://ies.ed.gov/ncer/funding/math_science/index.asp - VanDerHeyden, A., McLaughlin, T., Algina, J., & Snyder, P. (2012). Randomized evaluation of a supplemental grade-wide mathematics intervention.
*American Educational Research Journal, 49*(6), 1251–1284.CrossRefGoogle Scholar - Vuchinich, S., Flay, B. R., Aber, L., & Bickman, L. (2012). Person mobility in the design and analysis of cluster-randomized cohort prevention trials.
*Prevention Science, 13*(3), 300–313.CrossRefGoogle Scholar