Abstract
In this chapter, we focus on large-scale tests used to measure mathematical progress and proficiency. We examine the aims, capacity, and limitations of the tests, and what they can tell us about student performance. We reflect on whether we can rely on results from such tests to act as a catalyst for change, for dealing constructively with diversity, and enhancing social inclusion. By drawing on data from two countries, Australia and the USA, we enlarge our data base and also consider what can be learnt from international comparisons. In the Australian setting, we focus primarily on the National Assessment Program—Literacy and Numeracy (NAPLAN). For the American context, we have chosen to focus on the National Assessment of Educational Progress (NAEP). For additional context, we make limited reference to a broader range of tests. In the final section of the chapter, we take stock of what practitioners and researchers can learn from large-scale assessments and foreground steps that can help minimize the drawbacks and maximize the benefits of these data in ways that promote equity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Terminology (years or grades) as used in the tests.
- 2.
Grade 4 and 8 samples are particularly large because they are selected to be representative of each US state and then aggregated to be nationally representative. The grade 12 sample, in contrast is simply nationally representative and therefore smaller.
- 3.
These and other details are available at: http://nationsreportcard.gov/math_2009/about_math.asp, http://nationsreportcard.gov/math_2011/about_math.asp, and http://nces.ed.gov/nationsreportcard/tdw/sample_design/.
- 4.
The “Long-Term Trend” NAEP (administered periodically with far smaller sample sizes) is a more traditional, multiple-choice test that tracks US students’ mathematics and reading knowledge on the content considered important when it was begun in the early 1970s. We focus in this chapter on the more widely referenced and discussed “Main NAPLAN” data.
- 5.
See, e.g., 2012 NAPLAN National Report, retrieved from www.nap.edu.au/verve/_resources/NAPLAN_2012_National_Report.pdf.
- 6.
Although there are differences of opinions about appropriate categories and terms to use to describe various groups in the USA, we use NAEP’s school-reported categories and terms when describing NAEP results for racial and ethnic subgroups. Latino/a students are included as “Hispanic” and are generally not included in the “White” and “Black” categories. NAEP also includes a category for “2 or more races,” but we do not report on that relatively small category here.
- 7.
In this chapter, we use “sex differences” when it is clear that categorization is only based on biological factors. “Gender differences” are used when psychosocial or sociocultural factors may contribute to any difference found.
- 8.
The differences reported are somewhat smaller than the differences found on this measure between the non-LBOTE and LBOTE students discussed in the previous section.
- 9.
The first large-scale longitudinal survey of (Australian) Indigenous students [LSIC], also known as Footprints in time, began in 2008. The survey is conducted under the auspices of the Department of Families, Housing, Community Services and Indigenous Affairs. (Retrieved from http://www.fahcsia.gov.au/about-fahcsia/publications-articles/research-publications/longitudinal-data-initiatives/footprints-in-time-the-longitudinal-study-of-indigenous-children-lsic).
- 10.
- 11.
Propensity score matching allows researchers to match “treatment” and comparison groups on a large set of covariates in order to limit the potentially confounding effects of those covariates. Regression discontinuity designs are useful when a cut-score divides students (or others) into treatment and comparison groups, such as when schools use income, language ability, or a test score to determine which students should receive particular services or opportunities. See Stuart (2007) for further discussion of both of these methods. See Robinson (2010) for an application of regression discontinuity to the issue of mathematics test translations for English learners in the USA.
References
American Statistical Association. (2007). Using statistics effectively in mathematics education research. Alexandria, VA: Author. http://www.amstat.org/education/pdfs/UsingStatisticsEffectivelyinMathEdResearch.pdf. Retrieved from 19 June 2014.
Andrich, D. (1988). Rasch models for measurement. Beverly Hills: Sage.
Australian Curriculum Assessment Reporting and Curriculum Authority (ACARA). (2011a). Why NAP. ACARA. http://www.nap.edu.au/about/why-nap.html. Retrieved from 6 March 2012.
Australian Curriculum Assessment Reporting and Curriculum Authority (ACARA). (2011b). National Assessment Program: Standards. http://www.nap.edu.au/results-and-reports/how-to-interpret/standards.html. Retrieved from 7 Feb 2012.
Bachelard, M. (2011). Is this school incredibly good, or just incredible? The Age, March 6, 2011. (http://www.theage.com.au/national/education/is-this-school-incredibly-good-or-just-incredible-20110305-1bix3.html).
Carnoy, M., & Rothstein, R. (2013). What do international tests really show about U.S. student performance? Washington, DC: Economic Policy Institute.
Curriculum Corporation. (2006). Statements of learning for mathematics. http://www.mceetya.edu.au/verve/_resources/SOL_Maths_Copyright_update2008.pdf. Retrieved from 16 Dec 2010.
Figel, J. (2009). Preface. In National testing of pupils in Europe: Objectives, organisation and use of results. Brussels: Education, Audiovisual and Culture Executive Agency—Eurydice. http://www.eurydice.org. Retrieved from 27 March 2012.
Gutiérrez, R. (2008). A “gap-gazing” fetish in mathematics education? Problematizing research on the achievement gap. Journal for Research in Mathematics Education, 39(4), 357–364.
Heilig, J. V., & Jez, S. J. (2010). Teach for America: A review of the evidence. Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. (http://epicpolicy.org/publication/teach-for-america).
Hogrebe, M., & Tate, W. F. (2012). Place, poverty, and algebra: A statewide comparative spatial analysis of variable relationship. Journal of Mathematics Education at Teachers College, 3, 12–24.
Johnson, E. G. (1992). The design of the National Assessment of Educational Progress. Journal of Educational Measurement, 29(2), 95–110.
Johnson, E. G., & Rust, K. F. (1992). Population inferences and variance estimation for NAEP data. Journal of Educational Statistics, 17(2), 175–190.
Jones, L. V., & Olkin, I. (2004). The Nation’s report card: Evolution and perspectives. Bloomington: Phi Delta Kappa Educational Foundation.
Kenney, P. A., & Schloemer, C. G. (2001). Assessment of student achievement, overview. In S. Grinstein & S. I. Lipsey (Eds.), Encyclopedia of mathematics education (pp. 50–56). New York: RoutledgeFalmer.
Kloosterman, P., & Lester, F. (Eds.) (2007). Results and interpretation of the 2003 math assessment of NAEP. Reston: National Council of Teachers of Mathematics.
Laczko-Kerr, I., & Berliner, D. C. (6 Sept 2002). The effectiveness of “Teach for America” and other under-certified teachers on student academic achievement: A case of harmful public policy. Education Policy Analysis Archives, 10(37), 34–38. (http://epaa.asu.edu/ojs/article/view/316).
Lubienski, S. T. (2008). On “gap gazing” in mathematics education: The need for gaps analyses. Journal for Research in Mathematics Education, 39(4), 350–356.
Lubienski, S. T. (2011). Mathematics education and reform in Ireland: An outsider’s analysis of Project Maths. Bulletin of the Irish Mathematical Society, 67, 27–55.
Lubienski, S. T., & Crockett, M. (2007). NAEP mathematics achievement and race/ethnicity. In P. Kloosterman & F. Lester (Eds.), Results and interpretation of the 2003 math assessment of NAEP (pp. 227–260). Reston: National Council of Teachers of Mathematics.
Lubienski, S. T., & Gutiérrez, R. (2008). Bridging the “gaps” in perspectives on equity in mathematics education. Journal for Research in Mathematics Education, 39(4), 365–371.
Lubienski, S. T., Lubienski, C., & Crane, C. C. (2008). Achievement differences among public and private schools: The role of school climate, teacher certification and instruction. American Journal of Education, 151(1), 97–138.
Lubienski, S. T., & Ganley, C. M., & Crane, C. C. (2012). Unwarranted uncertainty: Gender patterns in early mathematical confidence, interest and achievement. Paper presented at the American Educational Research Association, Vancouver.
Lubienski, S. T., Robinson, J. P., Crane, C. C., & Ganley, C. M. (2013). Girls’ and boys’ mathematics achievement, affect and experiences: Findings from ECLS-K. Journal for Research in Mathematics Education, 44(4), 634–645.
Ministerial Council on Education, Employment, Training and youth Affairs. (Dec 2008). Melbourne declaration on educational goals for young Australians. http://www.mceecdya.edu.au/verve/_resources/national_declaration_on_the_educational_goals_for_young_australians.pdf. Retrieved from 16 Dec 2010.
Mullis, I. V. S., Martin, M. O., Foy, P., & Arora, A. (2012). TIMSS 2011 international results in mathematics. USA: TIMSS & PIRLS International Study Center and The Netherlands: IEA.
National Center for Education Statistics. (2006). Digest of education statistics: 2005 (NCES 2006-030). http://nces.ed.gov/programs/digest/d05/. Retrieved from 23 May 2007.
National Council of Teachers of Mathematics. (1989). Curriculum and evaluation standards for school mathematics. Reston: National Council of Teachers of Mathematics.
National Council of Teachers of Mathematics. (1991). Professional standards for teaching mathematics. Reston: National Council of Teachers of Mathematics.
National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston: National Council of Teachers of Mathematics. (http://standards.nctm.org/).
Polesel, J., Dulfer, N., & Turnbull, M. (2012). The experience of education: The impacts of high stake testing on students and their families. Rydalmere: University of Western Sydney, The Whitlam Institute. (http://www.whitlam.org/__data/assets/pdf_file/0008/276191/High_Stakes_Testing_Literature_Review.pdf).
Postlethwaite, T. N., & Kellaghan, T. (2009). National assessment of educational achievement. Paris: UNESCO, International Institute for Educational Planning. (http://www.iiep.unesco.org/fileadmin/user_upload/Info_Services_Publications/pdf/2009/EdPol9.pdf).
Reardon, S. F., & Robinson, J. P. (2007). Patterns and trends in racial/ethnic and socioeconomic academic achievement gaps. In H. A. Ladd & E. B. Fiske (Eds.), Handbook of research in education finance and policy (pp. 499–518). Mahwah: Erlbaum.
Robinson, J. P. (2010). The effects of test translation on young English learners’ mathematics performance. Educational Researcher, 39(8), 582–590.
Strauss, V. (2013). Atlanta test cheating: Tip of the iceberg? Washington Post (http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/04/01/atlanta-test-cheating-tip-of-the-iceberg/).
Stuart, E. A. (2007). Estimating causal effects using school-level data sets. Educational Researcher, 36(4), 187–198.
Thomson, S. (2010). Mathematics learning: What TIMSS and PISA can tell us about what counts for all Australian students. Paper presented at the ACER Research conference: Teaching mathematics? Make it count. (http://research.acer.edu.au/research_conference/RC2010/17august/6/).
Thomson, S., De Bortoli, L., Nicholas, M., Hillman, K., & Buckley, S. (2011). Challenges for Australian education. Results from PISA 2009. (http://www.acer.edu.au/documents/PISA-2009-Report.pdf).
Usiskin, Z. (1993). What changes should be made for the second edition of the NCTM Standards? University of Chicago School Mathematics Project Newsletter. Winter, 6–11.
Vinovskis, M. A. (1998). Overseeing the Nation’s report card. Paper prepared for National Assessment Governing board. (http://www.nagb.org/content/nagb/assets/documents/publications/95222.pdf).
Winerip, M. (30 March 2013). Ex-schools chief in Atlanta is indicted in testing scandal. The New York Times. (http://www.nytimes.com/2013/03/30/us/former-school-chief-in-atlanta-indicted-in-cheating-scandal.html).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Leder, G., Lubienski, S. (2015). Large-Scale Test Data: Making the Invisible Visible. In: Bishop, A., Tan, H., Barkatsas, T. (eds) Diversity in Mathematics Education. Mathematics Education Library. Springer, Cham. https://doi.org/10.1007/978-3-319-05978-5_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-05978-5_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-05977-8
Online ISBN: 978-3-319-05978-5
eBook Packages: Humanities, Social Sciences and LawEducation (R0)