Skip to main content

Impact of school inspections on improvement of schools—describing assumptions on causal mechanisms in six European countries

Educational Assessment, Evaluation and Accountability volume 25, pages 3–43 (2013)Cite this article

Abstract

School inspection is used by most European education systems as a major instrument for controlling and promoting the quality of schools. Surprisingly, there is little research knowledge about how school inspections drive the improvement of schools and which types of approaches are most effective and cause the least unintended consequences. The study presented in this paper uses interviews with inspection officials and a document analysis to reconstruct the “program theories” (i.e. the assumptions on causal mechanisms, linking school inspections to their intended outcomes of improved teaching and learning) of Inspectorates of Education in six European countries. The results section of the paper starts with a summary of the commonalities and differences of these six national inspection models with respect to standards and thresholds used, to types of feedback and reporting, and to the sanctions, rewards and interventions applied to motivate schools to improve. Next, the intermediate processes through which these inspection models are expected to promote good education (e.g. through actions of stakeholders) are explained. In the concluding section, these assumptions are critically discussed in the light of research knowledge.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1

Notes

  1. Austria is represented in this study by the regional education authority of the province of Styria which uses some room for maneuver in the Austrian centralist education legislation for developing a specific type of “team inspection”.

  2. http://www.onderwijsinspectie.nl/binaries/content/assets/Actueel_publicaties/2009/ICALT.pdf

References

  • Altrichter, H. (2010). Schul- und Unterrichtsentwicklung durch Datenrückmeldung. In H. Altrichter & K. Maag Merki (Eds.), Handbuch Neue Steuerung im Schulsystem (pp. S. 219–S. 254). Wiesbaden: VS Verlag.

    Chapter  Google Scholar 

  • Altrichter, H., & Maag Merki, K. (Eds.). (2010). Handbuch Neue Steuerung im Schulsystem (pp. S. 219–S. 254). Wiesbaden: VS Verlag.

    Google Scholar 

  • Altrichter, H., & Rürup, M. (2010). Schulautonomie und die Folgen. In H. Altrichter & K. Maag Merki (Eds.), Handbuch neue Steuerung im Schulwesen (pp. 111–144). Wiesbaden: VS.

    Chapter  Google Scholar 

  • Arnold, K.-H. (2007). Bildungspolitische, diagnostische und didaktische Bedingungen und Wirkungen von Schulleistungsevaluationen. Empirische Pädagogik, 21, 448–457.

    Google Scholar 

  • Belfield, C. R., & Levin, H. N. (2009). Market reforms in education. In G. Sykes, B. Schneider, D. N. Plank, & T. G. Ford (Eds.), Handbook of education policy research (pp. 513–527). New York: Routledge.

    Google Scholar 

  • Bonsen, M., & Gathen, J. (2004). Schulentwicklung und Testdaten. In H. G. Holtappels, K. Klemm, H. Pfeiffer, H.-G. Rolff, & R. Schulz-Zander (Eds.), Jahrbuch der Schulentwicklung. Band 13 (pp. S. 225–S. 252). Weinheim: Juventa.

    Google Scholar 

  • Brimblecombe, N., Shaw, M., & Ormston, M. (1996). Teachers’ intention to change practice as a result of OFSTED school inspections. Educational Management & Administration, 24(4), 339–354.

    Article  Google Scholar 

  • Brunsden, V., Davies, M., & Shevlin, M. (2006). Anxiety and stress in educational professionals in relation to Ofsted. Education Today, 56(1), 24–31.

    Google Scholar 

  • Buckley, J., & Schneider, M. (2003). Making the grade: comparing DC charter schools to other DC public schools. Educational Evaluation and Policy Analysis, 25, 203–216.

    Article  Google Scholar 

  • Chapman, C. (2001). Changing classrooms through inspections. School Leadership Management, 1(1), 59–73.

    Article  Google Scholar 

  • Coe, R. (2002). Evidence on the role and impact of performance feedback in schools. In A. J. Visscher & R. Coe (Eds.), School improvement through performance feedback (pp. S. 3–S. 26). Lisse: Swets & Zeitlinger.

    Google Scholar 

  • Cook-Sather, A. (2002). Authorizing student’s perspectives: towards trust. Dialogue and Change in Education Educational Researcher, 19(5), 2–14.

    Google Scholar 

  • Darling-Hammond, L. (2004). Standards, accountability and school reform. Teachers College Record, 106(6), 1047–1085.

    Article  Google Scholar 

  • De Wolf, I. F., & Janssens, F. J. G. (2007). Effects and side effects of inspections and accountability in education: an overview of empirical studies. Oxford Review of Education, 33(3), 379–396.

    Article  Google Scholar 

  • Dillon, S. (2011). The way I see it is…Whole-school evaluation in Irish post-primary schools from the perspectives of principals, teachers, parents and students. Unpublished Doctoral Dissertation. [Online] www.dcu.ie/doras.

  • DiMaggio, P. J., & Powell, W. W. (1983). The iron cage revisited: institutional isomorphism and collective rationality in organizational fields. American Sociological Review, 48(1), 147–160.

    Article  Google Scholar 

  • Doolaard, S., & Karstanje, P. (2001). Gebruik van publieke prestatie-indicatoren voor schoolverbetering. In A. B. Dijkstra, S. Karsten, R. Veenstra, & A. J. Visscher (Eds.), Het oog der natie: scholen op rapport; standaarden voor de publicatie van schoolprestaties (pp. 155–173). Assen: Koninklijke Van Gorcum BV.

    Google Scholar 

  • Dronkers, J., & Veenstra, R. (2001). Schoolprestatie-indicatoren in het voortgezet onderwijs: start, reacties en vervolg. In A. B. Dijkstra, S. Karsten, R. Veenstra, & A. J. Visscher (Eds.), Het oog der natie: scholen op rapport; standaarden voor de publicatie van schoolprestaties (pp. 21–36). Assen: Koninklijke Van Gorcum BV.

    Google Scholar 

  • Dubs, R. (2006). Bildungsstandards: Das Problem der schulpraktischen Umsetzung. Netzwerk–Die Zeitschrift für Wirtschaftsbildung, 1, 18–29.

    Google Scholar 

  • Ehren, M. C. M., & Visscher, A. J. (2006). Towards a theory on the impact of school inspections. British Journal of Educational Studies, 54(1), 51–72.

    Article  Google Scholar 

  • Ehren, M. C. M., & Visscher, A. J. (2008). The relationship between school inspections, school characteristics and school improvement. British Journal of Educational Studies, 56(2), 205–227.

    Article  Google Scholar 

  • Ehren, M. C. M., Leeuw, F. L., & Scheerens, J. (2005). On the impact of the Dutch educational supervision act; Analyzing assumptions concerning the inspection of primary education. American Journal of Evaluation, 26(1), 60–76.

    Article  Google Scholar 

  • Elmore, R. F., & Fuhrman, S. H. (2001). Research finds the false assumption of accountability. Phi Delta Kappan, 67(4), 9–14.

    Google Scholar 

  • Eurydice (2004). Evaluation of schools providing compulsory education in Europe. http://www.eurydice.org/portal/page/portal/Eurydice. Accessed January 2011.

  • Eurydice. (2007). School autonomy in Europe: policies and measures. Brussels: Eurydice.

    Google Scholar 

  • Faubert, V. (2009). School evaluation: current practices in OECD countries and a literature review. OECD Education Working Papers, No. 42, OECD Publishing. doi:10.1787/218816547156.

  • Gärtner, H., Füsemann, D., & Pant, H. A. (2009). Wirkungen von Schulinspektion aus Sicht betroffener Schulleitungen. Empirische Pädagogik, 23, 1–18.

    Google Scholar 

  • Geijsel, F., Berg, van den, R., & Sleegers, P. (1999). The innovative capacity of schools in primary education: a qualitative study. Qualitative studies in education, 12(2), 175–191.

    Google Scholar 

  • Geijsel, P., Sleegers, P. J. C., Stoel, R. D., & Kruger, M. L. (2009). The effect of teacher psychological and school organizational and leadership factors on teachers’ professional learning in Dutch schools. The Elementary School Journal, 109(4), 1–22.

    Article  Google Scholar 

  • Grabensberger, E., Freudenthaler, H. H., & Specht, W. (2008). Bildungsstandards: Testungen und Ergebnisrückmeldungen auf der achten Schulstufe aus der Sicht der Praxis. Graz: Bifie.

    Google Scholar 

  • Groß Ophoff, J., Koch, U., Helmke, A., & Hosenfeld, I. (2006). Vergleichsarbeiten für die Grundschule–und was diese daraus machen (können). Journal für Schulentwicklung, 10(4), 7–12.

    Google Scholar 

  • Gustafsson and Myrberg (in prep). School inspections of Swedish schools: a critical reflection on intended effects, causal mechanisms and methods. http://schoolinspections.eu/wp-content/uploads/downloads/2012/05/Sweden_PT.pdf. Accessed November 2011.

  • Hanushek, E. A., & Raymond, M. E. (2002). Lessons about the design of state accountability systems. Paper prepared for ‘Taking Account of Accountability: Assessing Policy and Politics’, Harvard University.

  • Hattie, J. (2009). Visible learning: a synthesis of over 800 meta-analysis relating to achievement. Milton Park: Routledge.

    Google Scholar 

  • Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.

    Article  Google Scholar 

  • Heubert, J. P., & Hauser, R. M. (Eds.). (1999). High stakes: testing for tracking, promotion, and graduation. Washington: National Academy Press.

    Google Scholar 

  • Hosenfeld, I., Groß Ophoff, J., & Koch, U. (2007). Vergleichsarbeiten in Klassenstufe 3 (VERA 3). Präsentation bei der 7. Tagung “Empiriegestützte Schulentwicklung” in Mainz. Landau: Universität Koblenz-Landau.

  • House, E., & Howe, K. (2000). Deliberative democratic evaluation. New Directions in Evaluation, 85, 3–26.

    Article  Google Scholar 

  • Hoxby, C. (Ed.). (2003). The economics of school choice. Chicago: University of Chicago Press.

    Google Scholar 

  • Hughes, G., Mears, R., & Winch, C. (1997). An inspector calls? Regulation and accountability in three public services. Policy and Politics, 25(3), 299–314.

    Article  Google Scholar 

  • Husfeldt, V. (2011). Wirkungen und Wirksamkeit der exzernen Schulevaluation. Überblick zum Stand der Forschung. Zeitschrift für Erziehungswissenschaft. doi:10.1007/s11618-011-0204-5.

  • Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979). Consequences of individual feedback on behaviour in organizations. Journal of Applied Psychology, 64(4), 349–371.

    Article  Google Scholar 

  • Karsten, S., & Visscher, A. J. (2001). Ervaringen met het openbaar maken van schoolprestaties in Engeland en Frankrijk. In A. B. Dijkstra, S. Karsten, R. Veenstra, & A. J. Visscher (Eds.), Het oog der natie: scholen op rapport (pp. 36–53). Assen: Koninklijke Van Gorcum.

    Google Scholar 

  • Kerr, S. (1975). On the folly of rewarding A, while hopping for B. The Academy of Management Journal, 18(4), 769–783.

    Article  Google Scholar 

  • Klerks, M. (submitted). The effect of school inspections: a systematic review. Oxford Review.

  • Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance. Psychological Bulletin, 119(2), 254–284.

    Article  Google Scholar 

  • Kogan, M., & Maden, M. (1999). An evaluation of evaluators: the OFSTED system of school inspection. In C. Cullingford (Ed.), An inspector calls; Ofsted and its effect on school standards (pp. 9–32). London: Kogan Page Limited.

    Google Scholar 

  • Koretz, D. M. (2003). Using multiple measures to address perverse incentives and score inflation. Educational Measurement, 22(2), 18–26.

    Article  Google Scholar 

  • Kotthoff, H. G., & Böttcher, W. (2010). Neue Formen der „Schulinspektion“: Wirkungshoffnungen und Wirksamkeit im Spiegel empirischer Bildungsforschung. In H. Altrichter, K. Maag Merki (Eds.), Handbuch Neue Steuerung in Schulsystem (pp. 295–325). Wiesbaden: VS Verlag für Sozialwissenschaften.

  • Kuper, H. (2005). Evaluation im Bildungssystem. Stuttgart: Kohlhammer.

    Google Scholar 

  • Leeuw, F. L. (2003). Reconstructing program theories: methods available and problems to be solved. American Journal of Evaluation, 24(1), 5–20.

    Google Scholar 

  • Leutner, D., Fleischer, J., Spoden, C., & Wirth, J. (2007). Schulrückmeldungen in landesweiten Lernstandserhebungen. Präsentation bei der 7. Tagung “Empiriegestützte Schulentwicklung” in Mainz. Landau: Universität Koblenz-Landau.

  • Luginbuhl, R., Webbink, D., & De Wolf, I. (2009). Do inspections improve primary school performance? Educational Evaluation and Policy Analysis, 31(3), 221–237.

    Google Scholar 

  • Maier, U. (2006). Können Vergleichsarbeiten einen Beitrag zur Schulentwicklung leisten? Journal für Schulentwicklung, 10(4), 20–28.

    Google Scholar 

  • Maier, U. (2007). Lehrereinschätzungen zu zentralen Tests und Leistungsrückmeldungen. Präsentation auf der 7. Tagung “Empiriegestützte Schulentwicklung” in Mainz. Landau: Universität Koblenz-Landau.

  • Malen, B. (1999). On rewards, punishments, and possibilities: teacher compensation as an instrument for education reform. Journal of Personnel Evaluation in Education, 12(4), 387–394.

    Article  Google Scholar 

  • Mason, R. O., & Mitroff, I. I. (1981). Challenging strategic planning assumptions; theory, cases and techniques. New York: Wiley.

    Google Scholar 

  • Matthews, P., & Sammons, P. (2004). Improvement through Inspection. London: Ofsted.

    Google Scholar 

  • McGarvey, N., & Stoker, G. (1999). Intervention, inspection, regulation and accountability in local government. DETR—Interim literature review. London: DETR.

    Google Scholar 

  • McNamara, G., & O’Hara, J. (2008). The importance of the concept of self-evaluation in the changing landscape of education policy. Studies in Educational Evaluation, 34, 173–179.

    Article  Google Scholar 

  • McNamara, G., & O’Hara, J. (2009). Where global meets local: contexts, constraints and consensus in school evaluation in Ireland. Sage international handbook of educational evaluation (pp. 273–291). London: Sage Publications.

    Book  Google Scholar 

  • Nevo, D. (2006). In S. J. Greene & M. Mark (Eds.), Evaluation in education. Sage handbook of evaluation I (pp. 441–460). London: Sage Publications.

    Google Scholar 

  • Nichols, S. L., Glass, G. V., & Berliner, D. C. (2006). High-stakes testing and student achievement: does accountability pressure increase student learning? Education Policy Analysis Archives, 14(1), Retrieved 14 November 2008 from http://epaa.asu.edu/epaa/v14n1.

  • Ofsted (2009). Parents’ perceptions of Ofsted’s work—a report of the NFER for Ofsted. London: Ofsted.

  • Peek, R. (2004). Qualitätsuntersuchung an Schulen zum Unterricht in Mathematik (QuaSUM). Empirische Pädagogik, 18, 82–114.

    Google Scholar 

  • Peek, R. (2006). Dateninduzierte Schulentwicklung. In H. Buchen & H.-G. Rolff (Eds.), Professionswissen Schulleitung (pp. 1343–1366). Weinheim: Beltz.

    Google Scholar 

  • Reback, R. (2005). Supply and demand in a public school choice program. New York: National Center for the Study of Privatization in Education.

    Google Scholar 

  • Rosenthal, L. (2004). Do school inspections improve school quality? Economics of Education Review, 23(2), 143–151.

    Article  Google Scholar 

  • Scheerens, J. (2009). Review and meta-analyses of school and teaching effectiveness. The Netherlands/department of Educational Organization and Management. www.iqb.hu-berlin.de/lehre/dateien/rapportScherens.pdf. Accessed November 2011.

  • Scheerens, J., Seidel, T., Witziers, B., Hendriks, M., & Doornekamp, G. (2005). Positioning and validating the supervision framework. Positioning the supervision frameworks for primary and secondary education of the Dutch Educational Inspectorate in current educational discourse and validating core indicators against the knowledge base of educational effectiveness research. Enschede/Kiel: University of Twente/IPN

  • Schildkamp, K., Visscher, A., & Luyten, H. (2009). The effects of the use of a school self-evaluation instrument. School Effectiveness and School Improvement, 20(1), 69–88.

    Article  Google Scholar 

  • Schwippert, K. (2004). Leistungsrückmeldungen an Grundschulen im Rahmen der internationalen Grundschul-Lese-Untersuchung (IGLU). Empirische Pädagogik, 18, 62–81.

    Google Scholar 

  • Stecher, B. M. (2002). Consequences of large-scale, high-stakes testing on school and classroom practices. Tests and their use in test-based accountability systems. In L. S. Hamilton, B. M., Stecher, S. P. Klein (Eds.), Making sense of Test-based accountability in education. Santa Monica: Rand cooperation. http://www.rand.org/pubs/monograph_reports/MR1554/.

  • Steffens, U. (2009). Lernstandserhebungen in den deutschen Ländern–Probleme und Perspektiven. Unv. Ms. Wiesbaden: Institut für Qualitätsentwicklung.

  • Toulmin, S. (1958). The uses of argument. Cambridge: University Press.

    Google Scholar 

  • Toulmin, S. (1964). The uses of argument. Cambridge: University Press.

    Google Scholar 

  • van Ackeren, I. (2003). Evaluation, Rückmeldung und Schulentwicklung. Münster: Waxmann.

    Google Scholar 

  • van Bruggen, J. C. (2010). Inspectorates of education in Europe; Some comparative remarks about their tasks and work. SICI report: www.sici-inspectorates.org.

  • van de Grift, W. (2007). Quality of teaching in four European countries: a review of the literature and application of an assessment instrument. Educational Research, 49(2), 127–152.

    Article  Google Scholar 

  • Visscher, A. J., & Coe, R. (2002). School improvement through performance feedback (27–39). Lisse: Swets & Zeitlinger.

  • Visscher, A. J., & Coe, R. (2003). School performance feedback systems: conceptualisation, analysis, and reflection. School Effectiveness and School Improvement, 14(3), 321–349.

    Article  Google Scholar 

  • Whitby, K. (2010). School inspection: recent experiences in high performing education systems; literature review. Reading: CfBT Education Trust.

    Google Scholar 

  • Wood, E. (2003). The power of pupil perspectives in evidence-based practice: the case of gender and underachievement. Research Papers in Education, 18(4), 365–383.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Education, 20 Bedford Way, WC1H 0AL, London, UK

    M. C. M. Ehren

  2. Johannes Kepler Universität Linz, 4040, Linz-Auhof Österreich, Austria

    H. Altrichter

  3. Dublin City University, School of Education Studies, Dublin 9, Ireland

    G. McNamara & J. O’Hara

Authors
  1. M. C. M. Ehren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. H. Altrichter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. G. McNamara
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. O’Hara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. C. M. Ehren.

Appendix

Appendix

Table 4 Summary of program theories of Inspectorates of Education in six European countries
Full size table

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Ehren, M.C.M., Altrichter, H., McNamara, G. et al. Impact of school inspections on improvement of schools—describing assumptions on causal mechanisms in six European countries. Educ Asse Eval Acc 25, 3–43 (2013). https://doi.org/10.1007/s11092-012-9156-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11092-012-9156-4

Keywords

  • School inspections
  • Accountability
  • External evaluation
  • Comparative research
  • School improvement
  • School effectiveness