Abstract
We discuss several challenges encountered during data collection and analyses in the Comparing Options in Secondary Mathematics: Investigating Curricula (COSMIC) project, a 3-year longitudinal comparative study of integrated and subject-specific curricula. Previously reported findings obscure the complexities associated with conducting well-designed studies of curricular effectiveness. In this chapter, we highlight key issues in our analyses of student and teacher data, and the construction of multilevel models of student learning outcomes. We discuss how these challenges were addressed, justify our decisions, and offer implications for conducting large-scale curriculum research in mathematics education.
The National Science Foundation under Grant No. (REC-0532214) supported the research reported in this chapter. The study was conducted as part of the Comparing Options in Secondary Mathematics: Investigating Curriculum (COSMIC) project, http://cosmic.missouri.edu. Any opinions, findings, and conclusions or recommendations expressed in this chapter are those of the authors and do not necessarily reflect the views of the National Science Foundation. Some of the results discussed herein were presented at the 2010, 2012, and 2013 annual meetings of the American Educational Research Foundation and/or published previously (e.g., Chávez, Tarr, Grouws, & Soria, 2015; Grouws et al., 2013; Tarr, Grouws, Chávez, & Soria, 2013). The authors wish to thank Robert Ankenmann, Angela Bowzer, Oscar Chavez, Dean Frerichs, Douglas A. Grouws, Melissa McNaught, Ira Papick, Greg Petroski, Robert Reys, Daniel Ross, Ruthmae Sears, and R. Didem Taylan for their assistance during various stages of the research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chávez, Ó., Papick, I., Ross, D. J., & Grouws, D. A. (2011). Developing fair tests for mathematics curriculum comparison studies: The role of content analyses. Mathematics Education Research Journal, 23(4), 397–416. doi:10.1007/s13394-011-0023-2.
Chávez, Ó., Tarr, J. E., Grouws, D. A., & Soria, V. (2015). Content organization, curriculum enactment, and student learning in the third-year high school mathematics curriculum: Results from the COSMIC project. International Journal of Mathematics and Science Educational Research, 13(1), 97–120. doi:10.1007/s10763-013-9443-7.
Elis, M. E., Malloy, C. E., Meece, J. L., & Sylvester, P. R. (2007). Convergence of observer ratings and student perceptions of reform teaching practices in sixth-grade mathematics classrooms. Learning Environments Research, 10(1), 1–15.
Frank, K. A. (2000). Impact of a confounding variable on a regression coefficient. Sociological Methods and Research, 29, 147–194.
Gelman, A., & Hill, J. (2006). Data analysis using regression and multilevel/hierarchical models. Cambridge, MA: Cambridge University Press.
Grouws, D. A., Tarr, J. E., Chávez, O., Sears, R., Soria, V., & Taylan, R. D. (2013). Curriculum and implementation effects on high-school students’ mathematics learning from curricula representing subject-specific and integrated content organizations. Journal for Research in Mathematics Education, 44(2), 416–463.
Harwell, M. R., Post, T. R., Maeda, Y., Davis, J. D., Cutler, A. L., & Kahan, J. A. (2007). Standards-based mathematics curricula and secondary students’ performance on standardized achievement tests. Journal for Research in Mathematics Education, 38(1), 71–101.
National Center for Education Statistics. (2007). Mapping 2005 state proficiency standards onto the NAEP scales (NCES 2007-482). U.S. Department of Education. Washington, DC: Author.
National Center for Education Statistics. (2010). Status and trends in the education of racial and ethnic minorities (NCES 2010-015). U.S. Department of Education. Washington, DC: Author.
National Council of Teachers of Mathematics. (1989). Curriculum and evaluation standards for school mathematics. Reston, VA: Author.
National Council of Teachers of Mathematics. (1991). Professional standards for teaching mathematics. Reston, VA: Author.
National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA: Author.
Post, T. R., Harwell, M. R., Davis, J. D., Maeda, Y., Cutler, A., & Andersen, E. (2008). Standards-based mathematics curricula and middle-grades students’ performance on standardized achievement tests. Journal for Research in Mathematics Education, 39(2), 184–212.
Tarr, J. E., Grouws, D. A., Chávez, O., & Soria, V. (2013). The effect of content organization and curriculum implementation on students’ mathematics learning in second-year high school courses. Journal for Research in Mathematics Education, 44(4), 683–729.
Tarr, J. E., Harwell, M., Grouws, D. A., Chávez, Ó., & Soria, V. (2013, April). Longitudinal effects of curriculum organization and implementation on high school students’ mathematics learning. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco.
Tarr, J. E., McNaught, M. D., & Grouws, D. A. (2012). The development of multiple measures of curriculum implementation in secondary mathematics classrooms: Insights from a three-year curriculum evaluation study. In I. Weiss, D. Heck, K. Chval, & S. Zeibarth (Eds.), Approaches to studying the enacted curriculum (pp. 89–115). Greenwich, CT: Information Age Publishing.
What Works Clearinghouse. (2008). Procedures and standards handbook (version 2). Retrieved from U.S. Department of Education website: http://nces.ed.gov/nationsreportcard/pdf/main2005/2007468.pdf
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Tarr, J.E., Soria, V. (2015). Challenges in Conducting Large-Scale Studies of Curricular Effectiveness: Data Collection and Analyses in the COSMIC Project. In: Middleton, J., Cai, J., Hwang, S. (eds) Large-Scale Studies in Mathematics Education. Research in Mathematics Education. Springer, Cham. https://doi.org/10.1007/978-3-319-07716-1_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-07716-1_5
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07715-4
Online ISBN: 978-3-319-07716-1
eBook Packages: Humanities, Social Sciences and LawEducation (R0)