Skip to main content

Advertisement

SpringerLink
Log in

To align or not to align: the enactment of accountability and data-use in disadvantaged school contexts

  • Published:
Educational Assessment, Evaluation and Accountability Aims and scope Submit manuscript

Abstract

Despite the growing number of researches about performance-based accountability (PBA) in education, there is still scarce evidence on the mediating role of subjective variables (e.g., perceived pressure and alignment to PBA mandates) in the enactment of PBA in socially disadvantaged contexts. This is paradoxical because marginalized schools are usually those that are on probation and have to cope with the threat of sanctions more frequently. Existing investigations on PBA enactment have put increasing attention to the role of situated and material contexts, but there is still limited knowledge on how subjective variables can mediate policy enactment processes and enable the adoption of different school responses. To address these gaps, the article aims to explore how the perceived accountability pressure, the school performative culture, and meaning-making processes at the school level are mediating the enactment of PBA policies in disadvantaged schools. At the theoretical level, the study is informed by sense-making and policy enactment frameworks. Methodologically speaking, the investigation uses a comparative case study approach based on two extreme cases, which have been selected on the basis of a factorial analysis that combines both survey and secondary data. The extreme cases represent two different scenarios, which, despite operating in similar situated contexts, are characterized by having opposite levels of perceived pressure and alignment with the performative culture. The case studies combine survey data (n = 39) with documentary analysis and semi-structured interviews with the management team and teachers (n = 7). The findings show that subjective variables, in interaction with other contextual factors, can exacerbate or inhibit PBA regulatory pressures and trigger diverging school responses.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. In 1982, the dictatorship enacted the School Performance Evaluation Test (PER, in Spanish), which evolved into the current Educational Quality Measurement System or SIMCE

  2. The data analyzed in this research comes from the REFORMED project This is an international project that analyzes the adoption and enactment of school autonomy with accountability policies in countries like Norway, the Netherlands, Spain and Chile. For more information about the project, see www.reformedproject.eu

  3. The following grades (4th, 6th and 8th) and subjects (language, mathematics, natural sciences, social sciences) are assessed in the census-based SIMCE test. For more information, see Ministerio de Educación (2016) Plan de evaluaciones nacionales e internacionales 2016–2020. Santiago: MINEDUC. Available at: https://curriculumnacional.mineduc.cl/614/articles-34980_recurso_1.pdf

  4. Further information about the rationale and the content of the questionnaire can be found in the questionnaire methodological note: Levatino, A. (2021, forthcoming). Surveying Principals and Teachers in the framework of the REFORMED Project: Methodological Insights into the Design of the Questionnaires. REFORMED Methodological Papers No. 2. www.reformedproject.eu

  5. There is a detailed explanation of how quadrants were obtained in Ferrer-Esteban, G. (2021, forthcoming). Sampling strategy. REFORMED Methodological Papers No. 3. Bellaterra: Universitat Autònoma de Barcelona. Available at: www.reformedproject.eu. For further information, see Verger et al. (2021, forthcoming)

  6. Since School A has fewer than 20 teachers, the sample of participants included all members of the teaching staff

  7. Teachers who taught in grades and subjects evaluated in the SIMCE test, and were available at the time of the visit to the school, were selected for the qualitative fieldwork

  8. The names of interviewees are preserved for privacy purposes. For this reason, pseudonyms are used to maintain the anonymity of the participants. The research follows the European code of conduct for research integrity and complies with the ethics review procedures from the European Research Council.

  9. Also known as profesor jefe, in Spanish. In Chile, the lead teacher is in charge of a grade. The lead teacher can teach various subjects beyond their own specialism. In the case of Joaquín, he taught languages and mathematics in a grade evaluated in the SIMCE test.

References

  • Achinstein, B., & Ogawa, R. (2006). (In) fidelity: What the resistance of new teachers reveals about professional principles and prescriptive educational policies. Harvard Educational Review, 76(1), 30–63.

  • Acuña, F., Assaél, J., Contreras, P., & Peralta, B. (2014). La traducción de los discursos de la política educativa en la cotidianeidad de dos escuelas municipales chilenas: La metáfora médica como vía de análisis. Psicoperspectivas, 13(1), 46–55.

    Article  Google Scholar 

  • Agencia de la Calidad (2019). Categorías de desempeño. Santiago. Retrieved from: https://www.agenciaorienta.cl/. Accessed 4 Feb 2020.

  • Armijo, M. (2019). Micro-política de una escuela marginalizada: entre inclusión y estandarización. Universitas, Revista de Ciencias Sociales y Humanas, 30, 191–209.

  • Assaél, J., Acuña, F., Contreras, P., & Corbalán, F. (2014). Transformaciones en la cultura escolar en el marco de la implementación de políticas de accountability en Chile. Un estudio etnográfico en dos escuelas clasificadas en recuperación. Estudios Pedagógicos, 40(2), 7–26.

    Article  Google Scholar 

  • Au, W. (2007). High-stakes testing and curricular control: a qualitative metasynthesis. Educational Researcher, 36(5), 258–267.

    Article  Google Scholar 

  • Ball, S. J. (1994). Education reform: a critical and post-structuralist approach. Buckingham: Open University Press.

    Google Scholar 

  • Ball, S. J., Maguire, M., Braun, A., & Hoskins, K. (2011). Policy actors: doing policy work in schools. Discourse: Studies in the cultural politics of education, 32(4), 625–639.

    Google Scholar 

  • Ball, S. J., Maguire, M., & Braun, A. (2012). How schools do policy: policy enactments in secondary schools. New York: Routledge.

    Google Scholar 

  • Bellei, C., & Vanni, X. (2015). The evolution of educational policy in Chile 1980-2014. In S. Schwartzman (Ed.), Education in South America. London (pp. 179–200). London: Bloomsbury Academic.

    Chapter  Google Scholar 

  • Berliner, D. (2011). Rational responses to high stakes testing: the case of curriculum narrowing and the harm that follows. Cambridge Journal of Education, 41(3), 287–302.

  • Braun, A., Ball, S. J., & Maguire, M. (2011a). Policy enactments in schools introduction: towards a toolbox for theory and research. Discourse: Studies in the Cultural Politics of Education, 32(4), 581–583.

    Google Scholar 

  • Braun, A., Ball, S. J., Maguire, M., & Hoskins, K. (2011b). Taking context seriously: towards explaining policy enactments in the secondary school. Discourse: Studies in the cultural politics of education, 32(4), 585–596.

    Google Scholar 

  • Burton, G. (2012). Hegemony and frustration: education policy making in Chile under the Concertación, 1990–2010. Latin American Perspectives, 39(4), 34–52.

    Article  Google Scholar 

  • Campos-Martínez, J., & Fernández, M. B. (2014). Resistiendo la emboscada neoliberal desde las escuelas: experiencias de organización docente contra las pruebas estandarizadas. Revista Docencia, 52, 72–87.

    Google Scholar 

  • Campos-Martínez, J., & Guerrero, P. (2016). Efectos indeseados de la medición de la calidad educativa en Chile. La respuesta de la sociedad civil. Cadernos CEDES, 36(100), 355–374.

    Article  Google Scholar 

  • Candido, H. H. D. (2019). Datafication in schools: enactments of quality assurance and evaluation policies in Brazil. International Studies in Sociology of Education, 29(1–2), 1–32.

  • Carrasco, A., & Fromm, G. (2016). How local market pressures shape leadership practices: evidence from Chile. Journal of Educational Administration and History, 48(4), 290–308.

    Article  Google Scholar 

  • Carrasco, A., Seppänen, P., Rinne, R., & Falabella, A. (2015). Educational accountability policy schemes in Chile and Finland. In P. Seppänen, A. Carrasco, M. Kalalahti, R. Rinne, & H. Simola (Eds.), Contrasting dynamics in education politics of extremes (pp. 53–80). Rotterdam: Sense Publishers.

    Google Scholar 

  • Carrasco, A., Gutiérrez, G., & Flores, C. (2017). Failed regulations and school composition: selective admission practices in Chilean primary schools. Journal of Education Policy, 32(5), 642–672.

    Article  Google Scholar 

  • Chubb, J. E., & Moe, T. M. (1990). Politics, markets, and America's schools. Washington: Brookings Institution Press.

  • Coburn, C. E. (2005). Shaping teacher sensemaking: school leaders and the enactment of reading policy. Educational Policy, 19(3), 476–509.

    Article  Google Scholar 

  • Corvalán, J., Carrasco, A., & García-Huidobro, J. E. (Eds.). (2016). Mercado escolar y oportunidad educacional: libertad, diversidad y desigualdad. Santiago: Ediciones Universidad Católica de Chile.

    Google Scholar 

  • Danermark, B., Ekstrom, M., Jakobsen, L., & Karlsson, J. C. (2002). Explaining society: Critical realism in the social sciences. London: Routledge.

    Google Scholar 

  • Diamond, J. B. (2012). Accountability policy, school organization, and classroom practice: partial recoupling and educational opportunity. Education and Urban Society, 44(2), 151–182.

    Article  Google Scholar 

  • Diaz Rios, C. (2018). The role of national paradigms in the translation of test-based accountability. Paper presented at the 2018 Conference of the Comparative and International Education Society: Remapping Global Education. North-South Dialogue. Mexico, march 25-29.

  • Elacqua, G., Jaimovich, A., & Román, A. (2019). The effects of accountability on the allocation of school resources: regression discontinuity evidence from Chile. IDB working paper series N° IDB-WP-1074. Washington: Inter-American Development Bank Education Division. https://doi.org/10.18235/0001979.

  • Eppel, E., Turner, D., & Wolf, A. (2011). Experimentation and learning in policy implementation: implications for public management. Working paper 11/04. Wellington: Institute of policy studies.

    Google Scholar 

  • Falabella, A. (2014). The performing school: the effects of Market & Accountability Policies. Education policy analysis archives, 22(70), 1–29.

    Google Scholar 

  • Falabella, A. (2016). ¿Qué aseguran las políticas de aseguramiento de la calidad?: un estudio de casos en distintos contextos escolares. Estudios Pedagógicos (Valdivia), 42(1), 107–126.

    Article  Google Scholar 

  • Falabella, A. (2020). The ethics of competition: accountability policy enactment in Chilean schools’ everyday life. Journal of Education Policy, 35(1), 23–45.

    Article  Google Scholar 

  • Falabella, A., & Opazo, C. (2014). Sistema de Aseguramiento de la Calidad y procesos de mejoramiento: una mirada desde la gestión educativa. Informe Ejecutivo. Santiago: CIDE.

    Google Scholar 

  • Ferrer-Esteban, G. (2021, forthcoming). Sampling strategy. REFORMED methodological papers no. 3. Cerdanyola del Vallès: Universitat Autònoma de Barcelona.

  • Flórez Petour, M. T., & Rozas Assael, T. (2019). Accountability from a social justice perspective: criticism and proposals. Journal of Educational Change, 1–26. https://doi.org/10.1007/s10833-019-09361-3.

  • Gerring, J. (2007). Case study research: Principles and practices. New York: Cambridge university press.

    Google Scholar 

  • Gysling, J. (2016). The historical development of educational assessment in Chile: 1810–2014. Assessment in education: principles. Policy & Practice, 23(1), 8–25.

    Google Scholar 

  • Hamilton, L. S., Stecher, B. M., & Klein, S. P. (2002). Making sense of test-based accountability in education. Santa Monica: Rand Corporation.

  • Hardy, I. (2015). Data, numbers and accountability: the complexity, nature and effects of data use in schools. British Journal of Educational Studies, 63(4), 467–486.

    Article  Google Scholar 

  • Hofflinger, A., & von Hippel, P. T. (2020). Missing children: How Chilean schools evaded accountability by having low-performing students miss high-stakes tests. Educational Assessment Evaluation and Accountability, 32, 127–152.

  • Jennings, J. L. (2010). School choice or schools’ choice? Managing in an era of accountability. Sociology of Education, 83(3), 227–247.

    Article  Google Scholar 

  • Keddie, A. (2014). ‘It's like Spiderman… with great power comes great responsibility’: school autonomy, school context and the audit culture. School Leadership & Management, 34(5), 502–517.

    Article  Google Scholar 

  • Koretz, D. (2008). Measuring up: what educational testing really tells us. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Koretz, D. (2017). The testing charade: pretending to make schools better. Chicago: University of Chicago Press.

    Book  Google Scholar 

  • Landri, P. (2018). Digital governance of education: technology, standards and Europeanization of education. London: Bloomsbury Publishing.

    Book  Google Scholar 

  • Levatino, A. (2021, forthcoming). Surveying Principals and Teachers in the framework of the REFORMED Project: Methodological insights into the design of the questionnaires. REFORMED methodological papers no. 2. Cerdanyola del Vallès: Universitat Autònoma de Barcelona.

  • Lingard, B., Martino, W., Rezai-Rashti, G., & Sellar, S. (2015). Globalizing educational accountabilities. New York: Routledge.

    Book  Google Scholar 

  • Lipman, P., & Haines, N. (2007). From accountability to privatization and African American exclusion: Chicago’s “renaissance 2010”. Educational Policy, 21(3), 471–502.

    Article  Google Scholar 

  • Lupton, R. (2004). Schools in disadvantaged areas: recognising context and raising quality. CASE Papers, 076. London: London School of Economics, Centre for Analysis of social exclusion.

    Google Scholar 

  • Lupton, R., & Thrupp, M. (2013). Headteachers’ readings of and responses to disadvantaged contexts: evidence from English primary schools. British Educational Research Journal, 39(4), 769–788.

    Google Scholar 

  • Maroy, C., & Pons, X. (2019). Accountability policies in education. Dordrecht: Springer.

    Book  Google Scholar 

  • Mayring, P. (2004). Qualitative content analysis. In U. Flick, E.v. Kardorff, & I. Steinke (Eds.), A companion to qualitative research (pp. 266–269). London: SAGE.

  • MINEDUC. (2019). Prioridades 2019 con IVE, SINAE, BÁSICA, MEDIA Y COMUNAL. Santiago: Ministry of Education. Retrieved from: https://www.junaeb.cl/ivev. Accessed 6 Feb 2020.

  • Mintrop, H. (2004). Schools on probation: How accountability works (and doesn’t work). New York: Teachers College Press.

    Google Scholar 

  • Mintrop, H., & Trujillo, T. (2007a). School Improvement under Test-Driven Accountability: a Comparison of High-and Low-Performing Middle Schools in California (CSE report 717). Los Angeles, CA: University of California, National Center for research on evaluation, standards, and student testing (CRESST).

    Google Scholar 

  • Mintrop, H., & Trujillo, T. (2007b). The practical relevance of accountability systems for school improvement: a descriptive analysis of California schools (CSE rep. No. 713). Los Angeles: University of California, National Center for research on evaluation, standards, and student testing.

    Google Scholar 

  • Mittleman, J., & Jennings, J. L. (2018). Accountability, achievement, and inequality in American public schools: a review of the literature. In B. Schneider (Ed.), Handbook of the sociology of education in the 21st century (pp. 475–492). Cham: Springer. https://doi.org/10.1007/978-3-319-76694-2.

    Chapter  Google Scholar 

  • Mizala, A., & Schneider, B. R. (2014). Negotiating education reform: teacher evaluations and incentives in C hile (1990–2010). Governance, 27(1), 87–109.

    Article  Google Scholar 

  • Montecinos, C., Madrid, R., Fernández, M. B., & Ahumada, L. (2014). A goal orientation analysis of teachers’ motivations to participate in the school self-assessment processes of a quality assurance system in Chile. Educational Assessment, Evaluation and Accountability, 26(3), 241–261.

    Article  Google Scholar 

  • Neave, G. (1998). The evaluative state reconsidered. European Journal of Education, 33(3), 265–284.

    Google Scholar 

  • Parcerisa, L., & Falabella, A. (2017). La consolidación del Estado evaluador a través de políticas de rendición de cuentas: trayectoria, producción y tensiones en el sistema educativo chileno. Education Policy Analysis Archives, 25(89), 1–24. https://doi.org/10.14507/epaa.25.3177.

  • Pino, M., Oyarzún, G., & Salinas, I. (2016). Crítica a la rendición de cuentas: Narrativa de resistencia al sistema de evaluación en Chile. Cadernos CEDES, 36(100), 337–354.

    Article  Google Scholar 

  • Rose, N., & Miller, P. (2008). Governing the present: administering economic, social and personal life. Cambridge: Polity Press.

    Google Scholar 

  • Russell, J. L., & Bray, L. E. (2013). Crafting coherence from complex policy messages: educators’ perceptions of special education and standards-based accountability policies. Education policy analysis archives, 21(12), 1–25.

    Google Scholar 

  • Schulte, B. (2018). Envisioned and enacted practices: educational policies and the ‘politics of use’in schools. Journal of Curriculum Studies, 50(5), 624–637.

    Article  Google Scholar 

  • Spillane, J. P. (2004). Standards deviations: how local schools misunderstand policy. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Steiner-Khamsi, G. (2014). Cross-national policy borrowing: understanding reception and translation. Asia Pacific Journal of Education, 34(2), 153–167.

    Article  Google Scholar 

  • Thiel, C., Schweizer, S., & Bellmann, J. (2017). Rethinking side effects of accountability in education: insights from a multiple methods study in four german school systems. Education Policy Analysis Archives, 25(93), 1–32.

  • Thrupp, M., & Easter, A. (2012). Research, analysis and insight into National Standards (RAINS) project. Hamilton: University of Waikato.

    Google Scholar 

  • Thrupp, M., & Lupton, R. (2006). Taking school contexts more seriously: the social justice challenge. British Journal of Educational Studies, 54(3), 308–328.

    Article  Google Scholar 

  • Valenzuela, J. P., & Montecinos, C. (2017). Structural reforms and equity in Chilean schools (pp. 1–25). Oxford Research Encyclopedia of Education. https://doi.org/10.1093/acrefore/9780190264093.013.108.

  • Valenzuela, J. P., Bellei, C., & Ríos, D. D. L. (2014). Socioeconomic school segregation in a market-oriented educational system. The case of Chile. Journal of Education Policy, 29(2), 217–241.

    Article  Google Scholar 

  • Verger, A., & Parcerisa, L. (2018). Test-based Accountability and the rise of regulatory governance in education: a review of global drivers. In A. Wilkins & A. Olmedo (Eds.), Education governance and social theory: Interdisciplinary approaches to research (pp. 139–158). London: Bloomsbury.

  • Verger, A., Bonal, X., & Zancajo, A. (2016a). Recontextualización de políticas y (cuasi) mercados educativos. Un análisis de las dinámicas de demanda y oferta escolar en Chile. Education Policy Analysis Archives/Archivos Analíticos de Políticas Educativas, 24, 1–27.

    Google Scholar 

  • Verger, A., Ferrer-Esteban, G., & Parcerisa, L. (2021, forthcoming). In and out of the ‘pressure cooker’: Schools’ varying responses to accountability and datafication. In S. Grek, C. Maroy & A. Verger (eds.) World Yearbook in Education 2021: Accountability and datafication in education. New York: Routledge.

  • Verger, A., Fontdevila, C., & Parcerisa, L. (2019). Reforming governance through policy instruments: How and to what extent standards, tests and accountability in education spread worldwide. Discourse: Studies in the Cultural Politics of Education, 40(2), 248–270. https://doi.org/10.1080/01596306.2019.1569882.

  • Verger, A., Fontdevila, C., & Zancajo, A. (2016). The privatization of education: a political economy of global education reform. New York: Teachers College Press.

  • Villalobos, C., & Quaresma, M. L. (2015). Sistema escolar chileno: características y consecuencias de un modelo orientado al mercado. Convergencia, 22(69), 63–84.

    Google Scholar 

  • Waldow, F. (2009). Undeclared imports: Silent borrowing in educational policy-making and research in Sweden. Comparative Education, 45(4), 477–494.

    Article  Google Scholar 

  • Weinstein, J., & Villalobos, C. (2016). Chile: La Experiencia De La Subvención Escolar Preferencial. In M. Souto Simão (Ed.), Transferencias Directas a Escuelas: Reflexiones Sobre Prácticas En América Latina (pp. 58–101). Buenos Aires: IIPE-Unesco.

    Google Scholar 

  • Weinstein, J., Marfán, J., & Horn, A. (2016). School leadership challenged by double accountability towards schools. In J. Easley & P. N. Tulowitzki (Eds.), Educational accountability: International perspectives on challenges and possibilities for school leadership (pp. 54–72). New York: Routledge.

    Google Scholar 

  • Yin, R. K. (2003). Case study research: design and methods. Thousand Oaks: Sage.

    Google Scholar 

  • Zancajo, A. (2017). Las Escuelas ante el mercado: análisis de las respuestas de la oferta escolar en un entorno de competencia: el caso de Chile (Doctoral dissertation). Cerdanyola del Vallès: Universitat Autònoma de Barcelona.

  • Zancajo, A. (2020). Schools in the marketplace: analysis of school supply responses in the Chilean education market. Educational Policy, 34(1), 43–64.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Sociology, Universitat Autònoma de Barcelona, Barcelona, Spain

    Lluís Parcerisa

Authors
  1. Lluís Parcerisa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lluís Parcerisa.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table 2 Cooperation between teachers
Full size table
Table 3 Trust in principals
Full size table
Table 4 Fairness
Full size table
Table 5 Validity and usefulness
Full size table
Table 6 Data use
Full size table
Table 7 Instructions to practice
Full size table
Table 8 Teaching to the test
Full size table
Table 9 Curricular narrowing and influence on pedagogy
Full size table
Table 10 Capacity to use data
Full size table
Table 11 Competition between teachers
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Parcerisa, L. To align or not to align: the enactment of accountability and data-use in disadvantaged school contexts. Educ Asse Eval Acc 33, 455–482 (2021). https://doi.org/10.1007/s11092-020-09341-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11092-020-09341-9

Keywords

Search

Navigation