Abstract
Accountability policies through standardized testing are widespread in diverse educational systems. Based on an ethnographic research study, we sought to understand how a learning assessment policy, used for over three decades in primary and lower secondary education, is lived and interpreted in daily school life in Chile. This article analyzes how learning assessment policies are re-contextualized (translated and interpreted) in three elementary and high schools located in poor neighborhoods with different performance categorization according to students’ test results. We conducted interviews and observations with teachers and school leadership team members. Results show that school actors experience the effects of testing both as a menace and as a pressure despite their different school categorization performance. They consider the tests a decontextualized and unjust measurement and therefore respond with similar strategies that seek to avoid these negative consequences. This study contributes to understanding the effects of tight accountability systems that attempt to place different levels of pressure through categorization.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Making data publicly available is not possible since it supposes a risk of revealing their identities.
Code availability
Not applicable.
Notes
This statement paraphrases the “Alto al Simce” activist campaign motto: what is essential is invisible to SIMCE.
References
Acuña, F., Assaél, J., Contreras, P., & Peralta, B. (2014). La traducción de los discursos de la política educativa en la cotidianeidad de dos escuelas municipales chilenas: La metáfora médica como vía de análisis. Psicoperspectivas, 13(1), 46–55. https://doi.org/10.5027/psicoperspectivas-Vol13-Issue1-fulltext-363
Anderson-Levitt, K. (2011). Anthropologies of education: A global guide to ethnographic studies of learning and schooling. Berghahn Books.
Asesorías para el Desarrollo. (2014). Estudio sobre el funcionamiento de los servicios de Asistencia Técnica Educativa (ATE) Chileno. http://www.registroate.cl/usuarios/admin3/doc/2015022612371908010.ESTUDIO_FUNCIONAMIENTO_SISTEMA_ATE_-_INFORME_FINAL___2014.pdf
Assaél, J., Contreras, P., Palma, E., Campos, J., Sisto, V., & Redondo, J. (2011). Ley SEP en escuelas municipales emergentes: ¿cambios en la identidad docente? I Congreso Internacional de Investigación en Educación, Cultura y Sociedad.
Cochran-Smith, M., Cummings Carney, M., Stringer Keefe, E., Burton, S., Chang, W. C., Fernández, M. B., Miller, A. F., Sánchez, J. G., & Baker, M. (2018). Reclaiming accountability in teacher education. New York: Teachers College Press.
Ball, S. (1987). La micropolítica de la escuela. Hacia una teoría de la organización escolar. Paidos.
Ball, S. (1993). What is Policy? Texts, trajectories and toolboxes. Discourse: Studies in the cultural politics of education, 13(2), 10–17. https://doi.org/10.1080/0159630930130203
Ball, S. (1997). Policy sociology and critical social research: A personal review of recent education policy and policy research. British Educational Research Journal, 23(3), 257–274. https://doi.org/10.1080/0141192970230302
Ball, S., Maguire, M., & Braun, A. (2012). How schools do policy: Policy enactments in secondary schools. Routledge.
Bellei, C. (2015). El Gran Experimento. Mercado y Privatización de la Educación Chilena. LOM Ediciones.
Bellei, C., & Vanni, X. (2015). Evolución de las políticas educacionales en Chile, 1980–2014. In C. Bellei (Ed.), El gran experimento. Mercado y privatización de la educación chilena (pp. 23–46). LOM Ediciones.
Bernstein, B. (1994). La estructura del discurso pedagógico. Morata.
Braun, A., Maguire, M., & Ball, S. (2010). Policy enactments in the UK secondary school: Examining policy, practice and school positioning. Journal of Education Policy, 25(4), 547–560. https://doi.org/10.1080/02680931003698544
Carrasco, A., & Fromm, G. (2016). How local market pressures shape leadership practices: Evidence from Chile. Journal of Educational Administration and History, 48(4), 290–308. https://doi.org/10.1080/00220620.2016.1210584
Cheung & Tai. (2023). The use of intercoder reliability in qualitative interview data analysis in science education. Research in Science and Technological Education, 41(3), 1155–1175.
CIDE (2012). IX Encuesta a Actores del Sistema Educativo. Recovered from http://www.cide.cl/documentos/Informe_IX_Encuesta_CIDE_2012.pdf
Collins, S., Reiss, M., & Stobart, G. (2010). What happens when high-stakes testing stops? Teachers’ perceptions of the impact of compulsory national testing in science of 11-year-olds in England and its abolition in Wales. Assessment in Education: Principles Policy and Practice, 17(3), 273–286.
Corvalán, J., & Román, M. (2012). La Permanencia de las Escuelas de Bajo Rendimiento Crónico en el Cuasi Mercado Educativo Chileno. Revista Uruguaya De Ciencia Política, 21(1), 43–64.
Cox, C. (2012). Política y políticas educacionales en Chile, 1990–2010. Revista Uruguaya De Ciencia Política, 21(1), 13–42.
Donoso, S. (2013). El Derecho a la educación. Santiago: Bravo y Allende Editores.
Falabella, A. (2015). ¿Qué Aseguran las Políticas de Aseguramiento de la Calidad? Un Estudio de Casos en Distintos Contextos Escolares. Estudios Pedagógicos, 42(1), 107–126. https://doi.org/10.4067/S0718-07052016000100007
Falabella, A. (2016). Do national test scores and quality labels trigger school self- assessment and accountability? A critical analysis in the Chilean context. British Journal of Sociology of Education, 37(5), 743–760. https://doi.org/10.1080/01425692.2014.976698
Falabella, A. (2020). The ethics of competition: Accountability policy enactment in Chilean schools’ everyday life. Journal of Education Policy, 35(1), 23–45. https://doi.org/10.1080/02680939.2019.1635272
Falabella, A., & Opazo, C. (2014). Sistema de Aseguramiento de la Calidad y Procesos de Mejoramiento: Una Mirada desde la Gestión Educativa. Santiago: CIDE.
Falabella, A., & De la Vega, L. (2016). Políticas de Responsabilización por Desempeño Escolar: Un Debate a Partir de la Literatura Internacional y el Caso Chileno. Estudios Pedagógicos, 42(2), 395–413. https://doi.org/10.4067/S0718-07052016000200023
Flórez, M. T. (2015). Systems, ideologies and history: A three-dimensional absence in the study of assessment reform processes. Assessment in Education: Principles, Policy and Practice, 22(1), 3–26. https://doi.org/10.1080/0969594X.2014.943153
Gerrard, J., & Farrel, L. (2013). ‘Peopling’ curriculum policy production: Researching educational governance through institutional ethnography and Bourdieuian field analysis. Journal of Education Policy, 28(1), 1–20. https://doi.org/10.1080/02680939.2012.664288
Goetz, J., & LeCompte, M. (1988). Etnografía y diseño cualitativo en investigación educativa. Morata.
Guber, R. (2011). La etnografía. Método, campo y reflexividad. Siglo XXI.
Gysling, J. (2016). The historical development of educational assessment in Chile: 1810–2014. Assessment in Education: Principles, Policy and Practice, 23(1), 8–25. https://doi.org/10.1080/0969594X.2015.1046812
Hammersley, M., & Atkinson, P. (1995). Ethnography: Principles in practice. Routledge.
Hardy, I. (2015). ‘I’m just a numbers person’: The complexity, nature and effects of the quantification of education. International Studies in Sociology of Education, 25(1), 20–37. https://doi.org/10.1080/09620214.2014.972971
Hickok, E. W. (1998). Higher standards for teacher training. Policy Review, 91, 6–9.
Himmel, E. & Comisión para el desarrollo y uso del sistema de medición de SIMCE. (2003). Evaluación de aprendizajes para una educación de calidad. Ministerio de Educación. https://hdl.handle.net/20.500.12365/17739
Hofflinger, A., & von Hippel, P. T. (2020). Missing children: How Chilean schools evaded accountability by having low-performing students miss high-stakes tests. Educational Assessment, Evaluation and Accountability, 32, 127–152. https://doi.org/10.1007/s11092-020-09318-8
Howell, A. (2017). ‘Because then you could never ever get a job!’: Children’s constructions of NAPLAN as high-stakes. Journal of Education Policy, 32(5), 564–587. https://doi.org/10.1080/02680939.2017.1305451
Hursh, D. (2013). Raising the stakes: High-stake testing and the attack on public education in New York. Journal of Education Policy, 28(5), 574–588. https://doi.org/10.1080/02680939.2012.758829
LeCompte, M. D., Preissle, J., & Tesch, R. (1993). Ethnography and qualitative design in educational research. Academic Press.
Lingard, B., Thompson, G., & Sellar, S. (2016). National testing in schools: An Australian assessment. Routledge.
Lunneblad, J., & Asplund Carlsson, M. (2012). Performativity as pretense: A study of testing practices in a compulsory school in Sweden. Ethnography and Education, 7(3), 297–309. https://doi.org/10.1080/17457823.2012.717198
Manzi, J., Bogolasky, F., Gutiérrez, G., Grau V., and Volante. P. (2014). Análisis Sobre Valoraciones, Comprensión y Uso del SIMCE por parte de Directores Escolares de Establecimientos Subvencionados. http://politicaspublicas.uc.cl/wp-content/uploads/2016/04/Informe-Final-F711269- Manzi.pdf
Ministerio de Educación. (2008). Ley de Subvención Escolar Preferencial (Nº 20.248). Santiago: Ministerio de Educación. https://obtienearchivo.bcn.cl/obtienearchivo?id=repositorio/10221/28046/2/BCN_Ley_SEP_actualizacion_modificaciones_Final.pdf
Ministerio de Educación. (2011). Ley de Aseguramiento de la Calidad de la Educación (N° 20.529). Santiago: Ministerio de Educación. https://www.bcn.cl/leychile/navegar?idNorma=1028635&idParte=9173238&idVersion=2019-04-25
Ministerio de Educación [Mineduc] (2014). Hacia un sistema completo y equilibrado de evaluación de los aprendizajes en Chile: Informe Equipo de Tarea para la Revisión del SIMCE. Ministerio de Educación. https://hdl.handle.net/20.500.12365/17445
Ministerio de Educación. (2015). Variación de matrícula y tasas de permanencia por sector. https://centroestudios.mineduc.cl/wp-content/uploads/sites/100/2017/06/Evidencias-final_julio_2015.pdf
Ministerio de Educación. (2016). Plan de evaluaciones nacionales e internacionales 2016–2020. http://www.curriculumenlineamineduc.cl/605/w3-article-34980.html
Montero, L., Cabalín, C., & Brossi, L. (2018). Alto al Simce: The campaign against standardized testing in Chile. Postcolonial Directions in Education, 7(2), 174–195.
Nichols, N., & Griffith, A. (2009). Talk, texts, and educational action: An institutional ethnography of policy in practice. Cambridge Journal of Education, 39(2), 241–255. https://doi.org/10.1080/03057640902902286
Olmedo, A. (2013). Policy-makers, market advocates and edu-businesses: New and renewed players in the Spanish education policy arena. Journal of Educational Policy, 28(1), 55–76. https://doi.org/10.1080/02680939.2012.689011
Parcerisa, L. (2021). To align or not to align: The enactment of accountability and data-use in disadvantaged school contexts. Educational Assessment, Evaluation and Accountability, 33(3), 455–482. https://doi.org/10.1007/s11092-020-09341-9
Rizvi, F., & Lingard, B. (2010). Globalizing educational policy. Routledge.
Rockwell, E. (2009). La experiencia etnográfica. Historia y cultura en los procesos educativos. Paidos.
Rojas, M. T., & Leyton, D. (2014). La Nueva Subjetividad Docente. Construcción de Subjetividades Docentes en los Inicios de la Implementación de la Subvención Escolar Preferencial en Chile. Estudios Pedagógicos, 40(1), 205–221.
Ruminot, C. (2017). Los Efectos Adversos de una Evaluación Nacional sobre las Prácticas de Enseñanza de las Matemáticas: El Caso de SIMCE en Chile”. Revista Iberoamericana De Evaluación Educativa, 10(1), 69–87.
Spillane, J., Diamond, J., Burch, P., Hallett, T., Jita, L., & Zoltners, J. (2002). Managing in the middle: School leaders and the enactment of accountability policy. Educational Policy, 16(5), 731–762.
Stobart, G. (2008). Testing times. The uses and abuses of assessment. Routledge.
Stone, D. (2012). Policy paradox: The art of political decision making. Norton.
Thiel, C., Schweitzer, S., & Bellmann, J. (2017). Rethinking side effects of accountability in education: Insights from multiple methods study in four German school systems. Education Policy Analysis Archives, 25(93), 1–27.
Thrupp, M. (2017). The vignettes as a whole: A reading of the vignettes from abroad. In T. Bibby, R. Lupton, & C. Raffo (Eds.), Responding to poverty and disadvantage in schools: A reader for teachers (pp. 170–177). Palgrave Macmillan.
Thrupp, M., Lupton, R., & Brown, C. (2007). Pursuing the contextualization agenda: Recent progress and future prospects. In T. Townsend (Ed.), International Handbook of school effectiveness and improvement: Review, reflection and reframing (pp. 111–126). Springer.
Verger, A., Bonal, X., & Zancajo, A. (2016). Recontextualización de políticas y (cuasi) mercados educativos Un análisis de las dinámicas de demanda y oferta escolar en Chile. Archivos Analíticos de Políticas Educativas, 24(27), 1–27. https://doi.org/10.14507/epaa.24.2098
Verger, A., Ferrer-Esteban, G., & Parcerisa, L. (2020). In and out of the ‘pressure cooker’: Schools’ varying responses to accountability and datafication. World Yearbook of Education, 2021, 241–261. https://doi.org/10.4324/9781003014164-17
Funding
Fondo Nacional de Desarrollo Científico y Tecnológico Regular grant no. 1160445; Support from ANID/PIA/Basal Funds for Centers of Excellence FB0003 is gratefully acknowledged; Fondo Nacional de Desarrollo Científico y Tecnológico Iniciación grant no. 11170663.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Consent to participate
Participants signed an informed consent that stated their identities and the schools would not be revealed in any way.
Competing interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Contreras, P., Santa Cruz, E., Assaél, J. et al. Re-contextualization of school quality assessment policies: an ethnography approach to SIMCE on Chilean disadvantaged schools. Educ Asse Eval Acc 36, 31–52 (2024). https://doi.org/10.1007/s11092-023-09425-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11092-023-09425-2