Skip to main content

Advertisement

Log in

School effectiveness research findings in the Portuguese speaking countries: Brazil and Portugal

  • Original Article
  • Published:
Educational Research for Policy and Practice Aims and scope Submit manuscript

Abstract

This paper provides findings of research on school effectiveness and discusses implications for evaluation in Brazil and Portugal. Most findings reported over the last decade have been published in Brazilian or Portuguese refereed journals. Thus, a brief literature review of such studies enables that knowledge to reach international scholars and researchers. The magnitude of school effects obtained from longitudinal and cross-sectional data modelling is presented and discussed. In particular, a value-added approach based on multilevel models is used to explore consistency across models with different controlling variables and across different curricular contents, and stability over time. The results show a great deal of regional disparities regarding educational outcomes that are related to pupils’ socioeconomic and prior achievement heterogeneity. In addition, evidence is given for stronger school effects in primary than in lower secondary education, larger amplitude of the variance partition coefficient in primary than in lower secondary education, stronger consistency of value-added estimates across models with different controlling variables and moderate consistency across different curricular contents. Weak-to-moderate stability of value-added estimates is also shown when yearly measured compared to when measured by cycles of education, and moderate-to-strong stability when measured in different curricular contents. Recommendations are outlined in terms of how the results could be used to mentoring evaluation in Portuguese speaking countries and enhance school improvement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Notes

  1. Agrupamento is a group of schools that consists of an organisational unit that aggregate two or more schools with its own administration and pedagogical bodies.

References

  • Aitkin, M., & Longford, N. (1986). Statistical modelling issues in school effectiveness studies. Journal of the Royal Statistical Society: Series A (Statistics in Society), 149, 1–43.

    Google Scholar 

  • Barbosa, M. E., & Fernandes, C. (2000). Modelo multinível: uma aplicação a dados de avaliação educacional. Estudos em Avaliaçáo Educacional, 22, 135–153.

    Google Scholar 

  • Barbosa, M. E., & Fernandes, C. (2001). A escola brasileira faz diferença? Uma investigação dos efeitos da escola na proficiência em matemática dos alunos da \(4^a\) série. In C. Franco (Ed.), Avaliaçáo, Ciclos e Promoçáo na Educaçáo (pp. 155–172). Porto Alegre: Artmed Editora.

  • Bird, S. M., Cox, D., Farewell, V. T., Goldstein, H., Holt, T., & Smith, P. C. (2005). Performance indicators: Good, bad, and ugly. Journal of the Royal Statistical Society: Series A (Statistics in Society), 168(1), 1–27.

    Article  Google Scholar 

  • Braun, H. (2013). Value-added modeling and the power of magical thinking. ENSAIO: Avaliação e Políticas Públicas em Educação, 21(78), in press.

  • Braun, H., Chudowsky, N., & Koenig, J. (2010). Getting value out of value-added. Social Sciences. Washington, DC: National Academies Press. Retrieved from http://216.78.200.159/Documents/RandD/Other/Getting Value out of Value-Added.pdf

  • Braun, H., & Wainer, H. (2007). Value-added modeling. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics: Psychometrics (Vol. 26, pp. 867–892). Amsterdam: Elsevier.

    Chapter  Google Scholar 

  • Brooke, N., & Bonamino, A. (Eds.). (2011). GERES 2005: Razões e Resultados de uma Pesquisa Longitudinal sobre Eficácia Escolar. Rio de Janeiro: Walprint.

    Google Scholar 

  • Clotfelter, C. T., Ladd, H. F., Vigdor, J. L., & Diaz, R. A. (2004). Do school accountability systems make it more difficult for low-performing schools to attract and retain high-quality teachers? Journal of Policy Analysis and Management, 23(2), 251–271.

    Article  Google Scholar 

  • Coleman, J., Campbell, E., Hobson, C., McPartland, J., Mood, A., Weinfeld, R., et al. (1966). Equality of Educational Opportunity. DC: Washington.

    Google Scholar 

  • Conselho de Ministros (2011). Portugal 2020 \({\vert }\) | Programa Nacional De Reformas.

  • Corcoran, T. B. (1985). Effective secondary schools. In A. M. J. Kyle (Ed.), Reaching for excellence: An effective schools sourcebook. Washington, D.C.: U.S. Government Printing Office.

    Google Scholar 

  • Cresswell, J. (2004). Schooling issues digest: school effectiveness. Canberra: Dept of Education, Science and Training.

  • Draper, D. (1995). Inference and hierarchical modeling in the social sciences. Journal of Educational and Behavioral Statistics, 20(2), 115–147.

    Google Scholar 

  • Fernandes, R. (2007). \(\acute{I}\)ndice de Desenvolvimento da Educação Básica (IDEB). Textos para Discussão; 26 (p. 26). Brasília: Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira.

  • Ferrão, M. E. (2008). Ainda os rankings e a estatística. Gazeta de Matemática, 155, 36–47.

    Google Scholar 

  • Ferrão, M. E. (2011). Adjusting for measurement error in longitudinal multilevel models. Dublin: ISI World Statistics Congress.

    Google Scholar 

  • Ferrão, M. E. (2012a). On the stability of value added indicators. Quality & Quantity, 46(2), 627–637.

    Article  Google Scholar 

  • Ferrão, M. E. (2012b). Eficácia Escolar no Brasil: uma década de estudos. In L. C. Freitas (Ed.), Avaliação e políticas públicas educacionais: ensaios contrarregulatórios em debate. Leituras Críticas: Campinas.

    Google Scholar 

  • Ferrão, M. E., Beltrão, K. I., Fernandes, C., Santos, D., Suarez, M., & Andrade, A. C. (2001). O SAEB-Sistema Nacional de Avaliação da Educação Básica: objectivos, características e contribuições na investigação da escola eficaz. Revista Brasileira de Estudos de População, 18, 111–130.

    Google Scholar 

  • Ferrão, M. E., Beltrão, K. I., & Santos, D. P. (2002). O impacto de políticas de não-repetência sobre o aprendizado dos alunos da \(4^{\rm a}\) série. Pesquisa e Planejamento Econômico, 32(3), 495–514.

  • Ferrão, M. E., & Couto, A. (2013a). Indicador de valor acrescentado, tópicos sobre consistência e estabilidade: uma aplicação ao Brasil. ENSAIO: Avaliação e Políticas Públicas em Educação, 21(78), in press.

  • Ferrão, M. E., & Couto, A. P. (2013b). The use of a school value-added model for educational improvement: a case study from the Portuguese primary education system. School Effectiveness and School Improvement, in press. doi:10.1080/09243453.2013.785436.

  • Ferrão, M. E., & Dias, M. (2010). A Retenção dos Alunos, a Composição da Escola e o Desempenho em Matemática: algumas pistas a partir do PISA [Student Repetition, School Composition and Performance in Mathematics: some clues based on PISA]. In G. Ramalho (Ed.), Impacto das Avaliações Internacionais nos Sistemas Educativos. Conselho Nacional de Educação: Lisboa.

    Google Scholar 

  • Ferrão, M. E., & Fernandes, C. (2003). O efeito-escola e a mudança -dá para mudar? Evidências da investigação brasileira. REICE-Revista Electrónica Iberoamericana sobre Calidad, Eficacia Y Cambio en Educación, 1(1). Retrieved from http://www.ice.deusto.es/rinace/reice/vol1n1/FerraoyFernandes.pdf

  • Ferrão, M. E., & Goldstein, H. (2009). Adjusting for measurement error in the value added model: Evidence from Portugal. Quality & Quantity, 43(6), 951–963.

    Article  Google Scholar 

  • Fletcher, P. (1998). Á Procura do Ensino Eficaz

  • Santiago, P., Roseveare, D., Amelsvoort, G., Manzi, J., & Matthews, P. (2009). Teacher evaluation in Portugal, OECD review. Paris: OECD.

  • Fontanive, N., & Klein, R. (2000). Uma Visão sobre O Sistema de Avaliação da Educação Básica do Brasil-SAEB. ENSAIO: Avaliação e Políticas Públicas em Educação, 8(29), 409–442.

  • Goldstein, H. (1997). Methods in school effectiveness research. School Effectiveness and School Improvement, 8(4), 369–395.

    Article  Google Scholar 

  • Goldstein, H., Browne, W., & Rasbash, J. (2002). Partitioning variation in multilevel models. Understanding Statistics, 1(4), 223–231.

    Article  Google Scholar 

  • Goldstein, H., & Spiegelhalter, D. (1996). League tables and their limitations: Statistical issues in comparisons of institutional performance. Journal of the Royal Statistical Society: Series A (Statistics in Society), 159, 385–443.

    Article  Google Scholar 

  • Gray, J., Jesson, D., Goldstein, H., Hedger, K., & Rasbash, J. (1995). A multi-level analysis of school improvement: Changes in schools’ performance over time. School Effectiveness and School Improvement, 6(2), 97–114.

    Article  Google Scholar 

  • Guldemond, H., & Bosker, R. J. (2009). School effects on students’ progress-A dynamic perspective. School Effectiveness and School Improvement, 20(2), 255–268.

    Article  Google Scholar 

  • Hanushek, E. A., Kain, J. F., & Rivkin, S. G. (2004). Why public schools lose teachers. Journal of Human Resources, 39(2), 326–354.

    Article  Google Scholar 

  • Hofstetter, C. H., & Alkin, M. C. (2003). Evaluation use revisited. In T. Kellaghan & D. L. Stufflebeam (Eds.), International handbook of educational evaluation (pp. 197–222). Dordrecht: Kluwer Academic Publishers.

    Chapter  Google Scholar 

  • Hopkins, D. (1994). School improvement in an era of change. In P. Ribbens & E. Burridge (Eds.), Improving Education Promoting Quality in Schools. London: Cassell.

    Google Scholar 

  • Ingersoll, R. M. (2003). The teacher shortage?: Myth or reality? Education Horizons, 81(3), 146–152.

    Google Scholar 

  • Ingersoll, R. M., & Perda, D. (2009). The Mathematics and Science Teacher Shortage: Fact and Myth (CPRE Research Report #RR-62) (p. 45). Pennsylvania.

  • Klein, R. (2006). Como está a educação no Brasil? O que fazer? ENSAIO: Aval. ENSAIO: Avaliação e Políticas Públicas em Educação, 14(51), 139–172.

  • Klein, R. (2013). Statistical Methods in Educational Evaluation: Item Response Theory (IRT) and large scale learning assessment in Brazil. In International Statistical Institute (Ed.), World Statistics Congress. Hong-Kong: International Statistical Institute.

  • Kyriakides, L., & Creemers, B. P. M. (2008). A longitudinal study on the stability over time of school and teacher effects on student outcomes. Oxford Review of Education, 34, 521–545.

    Article  Google Scholar 

  • Ladd, H. F., & Walsh, R. P. (2002). Implementing value-added measures of school effectiveness: Getting the incentives right. Economics of Education Review, 21(1), 1–17.

    Article  Google Scholar 

  • Laros, J. A., & Jesus, G. R. (2004). Eficácia escolar: Regressão multinível com dados de avaliação em larga escala. Avaliação Psicológica, 3(2), 93–106.

    Google Scholar 

  • Loura, L. C. C., Santos, J., Pereira, T., & Vitorino, A. (2012). Modelos para comparação estatística dos resultados académicos em escolas de contexto análogo (p. 16). Lisboa.

  • Martineau, J. A. (2006). Distorting value added: The use of longitudinal, vertically scaled student achievement data for growth-based, value-added accountability. Journal of Educational and Behavioral Statistics, 31(1), 35–62.

    Article  Google Scholar 

  • McPherson, A. (1993). Measuring added value in schools. Education Economics, 1(1), 43–51.

    Article  Google Scholar 

  • Ministério da Educação da República Democrática de Timor-Leste Plano da Educação 2013–2017 (2012).

  • Ministério da Educação e Ciência (2012). Despacho normativo no 13-A/2012. Diário da Républica 2asérie– no 109–5 de Junho de 2012.

  • Ministério da Educação e Cultura do Brasil Plano Nacional de Educação 2011–2020 (2011).

  • Ministério da Educaçao e Valorização dos Recursos Humanos da República de Cabo Verde Plano Estratégico para a Educação (2003).

  • Mortimore, P. (1991). The nature and findings of research on school effectiveness in the primary sector. In S. Riddell & S. Brown (Eds.), School Effectiveness Research: Its Messages for School Improvement (pp. 9–19). Edinburgh: HMSO.

  • Mortimore, P. (1998). The road to improvement: Reflections on school effectiveness. Lisse: Swets & Zeitlinger Publishers.

  • Mortimore, P., Sammons, P., Stoll, L., Lewis, D., & Ecob, R. (1988). School matters. Somerset: Open Books.

    Google Scholar 

  • Mortimore, P., & Whitty, G. (1997). Can school improvement overcome the effects of disadvantage?. London: Institute of Education.

    Google Scholar 

  • National Council of Education (2008). Judgement number 5/2008 on the External Evaluation of Schools. Lisbon: Diário da República 2asérie no 113 de 13 de Junho.

  • National Council of Education (2010). Judgement number 3/2010 on the External Evaluation of Schools (2007–2009). Lisbon: Diário da República 2asérie no 111 de 9 de Julho.

  • Nuttall, D., Goldstein, H., Prosser, R., & Rasbash, J. (1989). Differential school effectiveness. International Journal of Educational Research, 13(7), 689–825.

    Article  Google Scholar 

  • OECD. (2008). Measuring improvements in learning outcomes. Paris: OECD Publishing.

    Google Scholar 

  • Peng, J., Thomas, S., Yang, X., & Li, J. (2006). Developing school evaluation methods to improve the quality of schooling in China: a pilot “value added” study. Assessment in Education: Principles, Policy & Practice, 13(2), 135–154.

    Google Scholar 

  • Portela, M. C., & Thanassoulis, E. (2001). Decomposing school and school type efficiency. European Journal of Operational Research, 132(2), 114–130.

    Google Scholar 

  • Raudenbush, S. W., & Willms, J. D. (1995). The Estimation of School Effects. Journal of Educational and Behavioral Statistics, 20(4), 307–335.

    Google Scholar 

  • Reimers, F. (2003). The social context of education evaluation in Latin Ameria. In T. Kellaghan & D. L. Stufflebeam (Eds.), International handbook of educational evaluation (pp. 441–464). Dordrecht: Kluwer Academic Publishers.

    Chapter  Google Scholar 

  • Reynolds, D. (2010). Failure-Free Education? the past, present, and future of school effectiveness and school improvement. Abingdon: Routledge.

    Google Scholar 

  • Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15(3), 351–357.

    Article  Google Scholar 

  • Rubin, D. B., Stuart, E. A., & Zanutto, E. L. (2004). A potential outcomes view of value-added assessment in education. Journal of Educational and Behavioral Statistics, 29(1), 103–116.

    Article  Google Scholar 

  • Sammons, P. (2007). School effectiveness and equity: Making connections (pp. 1–77). Retrieved from http://www.cfbt.com/evidenceforeducation/pdf/FullLiteratureReview.pdf.

  • Sammons, P., Hillman, J., & Mortimore, P. (1995). Key Characteristics of Effective Schools: a review of school effectiveness research. London.

  • Sammons, P., Thomas, S., & Mortimore, P. (1997). Forging links: Effective schools and effective departments. London: Paul Chapman.

  • Santiago, P., Roseveare, D., Amelsvoort, G., Manzi, J., & Matthews, P. (2009). Teacher evaluation in Portugal, OECD review. Paris: OECD.

  • Saunders, L. (1999). A brief history of educational “value added”: How did we get to where we are? School effectiveness and school improvement, 10(2), 233–256.

    Article  Google Scholar 

  • Saunders, L. (2000). Understanding schools’ use of “value added” data: the psychology and sociology of numbers. Research Papers in Education, 15(3), 241–258.

    Article  Google Scholar 

  • Scheerens, J. (2001a). Introduction school effectiveness in developing countries. School Effectiveness and school improvement, 12(4), 353–358. doi:10.1076/sesi.12.4.353.3443.

  • Scheerens, J. (2001b). Monitoring school effectiveness in developing countries. School Effectiveness and School Improvement, 12(4), 359–384. doi:10.1076/sesi.12.4.359.3447

  • Scheerens, J., & Bosker, R. (1997). The foundations of educational effectiveness. Oxford: Pergamon.

    Google Scholar 

  • Soares, T. M. (2003). Influência do professor e do ambiente em sala de aula sobre a proficiência alcançada pelos alunos avaliados no Simave-2002. Estudos em Avaliação Educacional, 28, 103–123.

    Google Scholar 

  • Soares, J. F., Cesar, C. C., & Mambrini, J. (2001). Determinantes de Desempenho dos alunos do ensino básico brasileiro:Evidências do SAEB de 1997. In C. Franco (Ed.), Avaliação, Ciclos e Promoção na Educação (pp. 121–153). Porto Alegre: Artmed Editora.

    Google Scholar 

  • Strand, S. (2010). Do some schools narrow the gap? Differential school effectiveness by ethnicity, gender, poverty, and prior achievement. School Effectiveness and School Improvement, 21(3), 289–314.

    Article  Google Scholar 

  • Teddlie, C., & Reynolds, D. (2000). The international handbook of school effectiveness research. London and New York: Routledge/Falmer.

    Google Scholar 

  • Teddlie, C., & Stringfield, S. (1993). Schools make a difference: Lessons learned from a 10-year study of school effects. New York: Teachers College Press.

    Google Scholar 

  • UNESCO. (2012). Perfil EPT de Angola. Dakar.

  • United Nations. (2007). Plano Quadro das Nações Unidas para a ajuda ao desenvolvimento (PNUAD) 2008–2012. Guiné-Bissau.

  • United Nations. (2013). World Population Prospects: The 2010 Revision. Population Division of the Department of Economic and Social Affairs of the United Nations Secretariat, Retrieved from http://esa.un.org/unpd/wpp/index.htm.

  • United Nations Development Programme. (2011). Human development report 2011-Sustainability and equity: A better future for all. New York: Palgrave Macmillan.

    Google Scholar 

  • United Nations Development Programme. (2012). Human Development Index 2012. Washington, D.C. Retrieved February 25, 2013, from http://hdr.undp.org.

  • Vicente, P. (2007). Plano Amostral do Projecto 3EM-Eficácia Escolar no Ensino da Matemática [Sampling Design of 3EM Survey-School Effectiveness in Maths]. In M. E. Ferrão, C. Nunes, & C. Braumann (Eds.), Estatística: Ciência Interdisciplinar (pp. 849–858). Lisboa: Sociedade Portuguesa de Estatistica.

  • Weiss, C. H. (1977). Using social research in public policy making. Lexington, MA: Books, Lexington.

    Google Scholar 

  • Weiss, C. H. (1979). The many meanings of research utilization. Public Administration Review, 39(5), 426–431.

    Google Scholar 

  • Willms, J. D. (2008). Seven key issues for assessing “value added” in education. DC: Washington.

    Google Scholar 

  • Yu, G. (2007). Research evidence of school effectiveness in sub-saharan Africa. Retrieved from http://www.edqual.org/publications/workingpaper/edqualwp7.pdf.

Download references

Acknowledgments

The author gratefully acknowledges FCT—Fundação para a Ciência e Tecnologia (reference PEst-OE/EGE/UI0491/2011) for financial support, Alcino Couto and two anonymous referees for helpful comments and suggestions. The longitudinal survey 3EM was made possible by funding from the Portuguese Ministry of Science Technology and Higher Education and by the Calouste Gulbenkian Foundation.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Maria Eugénia Ferrão.

Additional information

This is an extended version of the paper presented at the 2012 World Educational Research Association focal meeting on the \(3\mathrm{rd}\) December in Sydney.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ferrão, M.E. School effectiveness research findings in the Portuguese speaking countries: Brazil and Portugal. Educ Res Policy Prac 13, 3–24 (2014). https://doi.org/10.1007/s10671-013-9151-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10671-013-9151-7

Keywords

Navigation