Data-Driven Education Reform: A New Pseudoscience
The underlying logic and assumptions behind the testing and assessment craze that has swept through education is examined, and the erroneous chain of reasoning based on tests and related metrics that leads to dangerous education policy decisions is uncovered. Many of the “data-driven” education policies and reforms are shown to resemble pseudoscience more than real science because they are missing one or more of the essential elements of science identified in Chapter 1. Examples discussed are value-added models for teacher effectiveness, measurements of adequate yearly progress in schools, college admissions based on SAT scores, and assessment practices in higher education. It is shown that all of these practices fail to provide causal explanations for student learning. The underlying reasons for why some teachers are more effective than others, why some schools succeed while others fail, why some students succeed in college while others drop out, why some college programs achieve their learning outcomes while others do not, are left unexamined. Instead these metrics provide tautologies—successful programs, teachers, schools, and students have successful outcomes.
- 1.Michael Winerip, “Evaluating New York Teachers, Perhaps the Numbers Do Lie,” New York Times, March 6, 2011, http://www.nytimes.com/2011/03/07/education/07winerip.html?pagewanted=all.
- 2.Eva L. Baker, et al., “Problems with the Use of Student Test Scores to Evaluate Teachers,” EPI Briefing Paper #278 (Washington, DC: Economic Policy Institute, 2010). http://s2.epi.org/files/page/-/pdf/bp278.pdf.
- 3.Eric A. Hanushek, and Steven G. Rivkin, “How to Improve the Supply of High Quality Teachers,” Brookings Papers on Education Policy, (Washington, DC: Brookings Institution Press, 2004) 7–25. http://muse.jhu.edu/journals/pep/summary/v2004/2004.1hanushek.html.
- 4.Richard Rothstein, “Comments” Brookings Papers on Education Policy, (Washington, DC: Brookings Institution Press, 2004) 25–39. http://muse.jhu.edu/journals/brookings_papers_on_education_policy/v2004/2004.1rothstein.html.
- 5.Edward H. Haertel, “Reliability and Validity of Inferences About Teachers Based on Student Test Scores,” 14th William H. Angoff Memorial Lecture on March 22, 2013 (Princeton: Educational Testing Service, 2013). https://www.ets.org/Media/Research/pdf/PICANG14.pdf.Google Scholar
- 6.Jason Felch, Jason Song, and Doug Smith, “Who’s teaching L.A.’s kids?” LA Times, August 14, 2010, http://www.latimes.com/local/la-meteachers-value-20100815-story.html#page=1.
- 7.John Ewing, “Mathematical Intimidation: Driven by Data,” Notices of the AMS 58, no. 5 (2010): 667–673.http://www.ams.org/notices/201105/rtx110500667p.pdf.
- 8.“Los Angeles Teacher Ratings,” LA Times, accessed May 1, 2018, http://projects.latimes.com/value-added/faq/#what_is_value_added.Google Scholar
- 9.Robert Gordon, Thomas J. Kane, and Douglas O. Staiger, “Identifying Effective Teachers Using Performance on the Job,” The Hamilton Project (Washington, DC: Brookings Institution, 2006). Accessed May 1, 2018, https://www.brookings.edu/research/identifying-effectiveteachers-using-performance-on-the-job/.
- 10.Cathy O’Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy (New York: Crown Publishing, 2016).Google Scholar
- 11.Public Law 107 – 110, An Act “To close the achievement gap with accountability, flexibility, and choice, so that no child is left behind,”Enacted by the Senate and House of Representatives of the United States of America in Congress Assembled, January 8, 2002, https://www2.ed.gov/policy/elsec/leg/esea02/107-110.pdf.
- 12.Diane Ravitch, The Death and Life of the Great American School System: How Testing and Choice Are Undermining Education (New York: Basic Books, 2010).Google Scholar
- 13.Emily Merwin, John Perry, and M. B. Pell, “Suspect scores, substantiated cheating,” Atlanta Journal-Constitution, September 22, 2012, http://www.ajc.com/news/cheating-our-children/district-bios/.
- 14.Erin Fuchs, “An epic cheating trial has come to an end, and teachers could go to jail for 20 years,” Business Insider, April 1, 2015, http://www.businessinsider.com/atlanta-teachers-convicted-of-racketeering-2015-4.
- 15.Donald T. Campbell, “Assessing the Impact of Planned Social Change,” Occasional Paper Series Paper #8, (Dartmouth College: The Public Affairs Center, 1976). http://portals.wi.wur.nl/files/docs/ppme/Assessing_impact_of_planned_social_change.pdf.
- 16.“Best Colleges: U. S. News and World Report Rankings,” U. S. News & World Report, accessed May 1, 2018, http://colleges.usnews.rankingsandreviews.com/best-colleges.
- 17.Princeton Review, The Best 379 Colleges, 2015 Edition (New York: Random House, 2014).Google Scholar
- 18.William C. Hiss and Valerie W. Franks, “Defining Promise: Optional Standardized Testing Policies in American College and University Admissions” (Arlington, VA: National Association for College Admissions Counseling, 2014). http://www.nacacnet.org/research/researchdata/nacac-research/Documents/DefiningPromise.pdf.
- 19.Middle States Commission on Higher Education, Student Learning Assessment: Options and Resources, 2nd edition (Philadelphia: Middle States Commission on Higher Education, 2007). accessed August 6, 2014, http://www.msche.org/publications/SLA_Book_0808080728085320.pdf.
- 20.Han Su Lee and Jongwon Park, “Deductive Reasoning to Teach Newton’s Law of Motion,” International Journal of Science Mathematics Education 11, (2013): 1391–1414.Google Scholar
- 21.U. S. Bureau of Labor Statistics, “Earnings and Employment rates by educational attainment,” (Washington, DC: U. S. Department of Labor, 2013): Fig. 1, accessed August 6, 2014, http://www.bls.gov/emp/ep_chart_001.htm.
- 22.U. S. Bureau of Labor Statistics, “Marriage and Divorce: Patterns by gender, race and educational attainment,” Monthly Labor Review,(October 2013), http://www.bls.gov/opub/mlr/2013/article/pdf/marriage-and-divorce-patterns-by-gender-race-and-educational-attainment.pdf.
- 23.S. Jay Olshansky, et al. “Differences In Life Expectancy Due To Race And Educational Differences Are Widening, And Many May Not Catch Up” Health Affairs 31, no. 8 (August 2012): 1803–1813. http://content.healthaffairs.org/content/31/8/1803.abstract.
- 24.The National Commission on Excellence in Education, “A Nation at Risk: The Imperative for Educational Reform,” A Report to the Nation and the Secretary of Education, (Washington, DC: United States Department of Education, 1983). https://www2.ed.gov/pubs/NatAtRisk/index.html.
- 25.Karl Weber, ed., Waiting for Superman: How We Can Save America’s Failing Public Schools, Participant Media (New York: PublicAffairs, 2010).Google Scholar
- 26.Diane Ravitch, Reign of Error: The Hoax of the Privatization Movement and the Danger to America’s Public Schools (New York: Alfred A. Knopf, 2013).Google Scholar
- 27.Claudia M. Mueller and Carol Dweck, “Praise for Intelligence Can Undermine Children’s Motivation and Performance,” Journal of Personality and Social Psychology 75 (1998): 33–52.Google Scholar
- 28.Carol Dweck, Mindset: The New Psychology of Success (New York: Random House, 2006).Google Scholar