Abstract
School inspection is used by most European education systems as a major instrument for controlling and promoting the quality of schools. Surprisingly, there is little research knowledge about how school inspections drive the improvement of schools and which types of approaches are most effective and cause the least unintended consequences. The study presented in this paper uses interviews with inspection officials and a document analysis to reconstruct the “program theories” (i.e. the assumptions on causal mechanisms, linking school inspections to their intended outcomes of improved teaching and learning) of Inspectorates of Education in six European countries. The results section of the paper starts with a summary of the commonalities and differences of these six national inspection models with respect to standards and thresholds used, to types of feedback and reporting, and to the sanctions, rewards and interventions applied to motivate schools to improve. Next, the intermediate processes through which these inspection models are expected to promote good education (e.g. through actions of stakeholders) are explained. In the concluding section, these assumptions are critically discussed in the light of research knowledge.
Similar content being viewed by others
Notes
Austria is represented in this study by the regional education authority of the province of Styria which uses some room for maneuver in the Austrian centralist education legislation for developing a specific type of “team inspection”.
Data collection was carried out by a team of researchers, including (in addition to the authors of this paper) Peter Tymms, Karen Jones, Jan-Eric Gustafsson, Eva Myrberg, Gerry Conyngham, David Kemethofer and David Greger.
References
Altrichter, H. (2010). Schul- und Unterrichtsentwicklung durch Datenrückmeldung. In H. Altrichter & K. Maag Merki (Eds.), Handbuch Neue Steuerung im Schulsystem (pp. S. 219–S. 254). Wiesbaden: VS Verlag.
Altrichter, H., & Maag Merki, K. (Eds.). (2010). Handbuch Neue Steuerung im Schulsystem (pp. S. 219–S. 254). Wiesbaden: VS Verlag.
Altrichter, H., & Rürup, M. (2010). Schulautonomie und die Folgen. In H. Altrichter & K. Maag Merki (Eds.), Handbuch neue Steuerung im Schulwesen (pp. 111–144). Wiesbaden: VS.
Arnold, K.-H. (2007). Bildungspolitische, diagnostische und didaktische Bedingungen und Wirkungen von Schulleistungsevaluationen. Empirische Pädagogik, 21, 448–457.
Belfield, C. R., & Levin, H. N. (2009). Market reforms in education. In G. Sykes, B. Schneider, D. N. Plank, & T. G. Ford (Eds.), Handbook of education policy research (pp. 513–527). New York: Routledge.
Bonsen, M., & Gathen, J. (2004). Schulentwicklung und Testdaten. In H. G. Holtappels, K. Klemm, H. Pfeiffer, H.-G. Rolff, & R. Schulz-Zander (Eds.), Jahrbuch der Schulentwicklung. Band 13 (pp. S. 225–S. 252). Weinheim: Juventa.
Brimblecombe, N., Shaw, M., & Ormston, M. (1996). Teachers’ intention to change practice as a result of OFSTED school inspections. Educational Management & Administration, 24(4), 339–354.
Brunsden, V., Davies, M., & Shevlin, M. (2006). Anxiety and stress in educational professionals in relation to Ofsted. Education Today, 56(1), 24–31.
Buckley, J., & Schneider, M. (2003). Making the grade: comparing DC charter schools to other DC public schools. Educational Evaluation and Policy Analysis, 25, 203–216.
Chapman, C. (2001). Changing classrooms through inspections. School Leadership Management, 1(1), 59–73.
Coe, R. (2002). Evidence on the role and impact of performance feedback in schools. In A. J. Visscher & R. Coe (Eds.), School improvement through performance feedback (pp. S. 3–S. 26). Lisse: Swets & Zeitlinger.
Cook-Sather, A. (2002). Authorizing student’s perspectives: towards trust. Dialogue and Change in Education Educational Researcher, 19(5), 2–14.
Darling-Hammond, L. (2004). Standards, accountability and school reform. Teachers College Record, 106(6), 1047–1085.
De Wolf, I. F., & Janssens, F. J. G. (2007). Effects and side effects of inspections and accountability in education: an overview of empirical studies. Oxford Review of Education, 33(3), 379–396.
Dillon, S. (2011). The way I see it is…Whole-school evaluation in Irish post-primary schools from the perspectives of principals, teachers, parents and students. Unpublished Doctoral Dissertation. [Online] www.dcu.ie/doras.
DiMaggio, P. J., & Powell, W. W. (1983). The iron cage revisited: institutional isomorphism and collective rationality in organizational fields. American Sociological Review, 48(1), 147–160.
Doolaard, S., & Karstanje, P. (2001). Gebruik van publieke prestatie-indicatoren voor schoolverbetering. In A. B. Dijkstra, S. Karsten, R. Veenstra, & A. J. Visscher (Eds.), Het oog der natie: scholen op rapport; standaarden voor de publicatie van schoolprestaties (pp. 155–173). Assen: Koninklijke Van Gorcum BV.
Dronkers, J., & Veenstra, R. (2001). Schoolprestatie-indicatoren in het voortgezet onderwijs: start, reacties en vervolg. In A. B. Dijkstra, S. Karsten, R. Veenstra, & A. J. Visscher (Eds.), Het oog der natie: scholen op rapport; standaarden voor de publicatie van schoolprestaties (pp. 21–36). Assen: Koninklijke Van Gorcum BV.
Dubs, R. (2006). Bildungsstandards: Das Problem der schulpraktischen Umsetzung. Netzwerk–Die Zeitschrift für Wirtschaftsbildung, 1, 18–29.
Ehren, M. C. M., & Visscher, A. J. (2006). Towards a theory on the impact of school inspections. British Journal of Educational Studies, 54(1), 51–72.
Ehren, M. C. M., & Visscher, A. J. (2008). The relationship between school inspections, school characteristics and school improvement. British Journal of Educational Studies, 56(2), 205–227.
Ehren, M. C. M., Leeuw, F. L., & Scheerens, J. (2005). On the impact of the Dutch educational supervision act; Analyzing assumptions concerning the inspection of primary education. American Journal of Evaluation, 26(1), 60–76.
Elmore, R. F., & Fuhrman, S. H. (2001). Research finds the false assumption of accountability. Phi Delta Kappan, 67(4), 9–14.
Eurydice (2004). Evaluation of schools providing compulsory education in Europe. http://www.eurydice.org/portal/page/portal/Eurydice. Accessed January 2011.
Eurydice. (2007). School autonomy in Europe: policies and measures. Brussels: Eurydice.
Faubert, V. (2009). School evaluation: current practices in OECD countries and a literature review. OECD Education Working Papers, No. 42, OECD Publishing. doi:10.1787/218816547156.
Gärtner, H., Füsemann, D., & Pant, H. A. (2009). Wirkungen von Schulinspektion aus Sicht betroffener Schulleitungen. Empirische Pädagogik, 23, 1–18.
Geijsel, F., Berg, van den, R., & Sleegers, P. (1999). The innovative capacity of schools in primary education: a qualitative study. Qualitative studies in education, 12(2), 175–191.
Geijsel, P., Sleegers, P. J. C., Stoel, R. D., & Kruger, M. L. (2009). The effect of teacher psychological and school organizational and leadership factors on teachers’ professional learning in Dutch schools. The Elementary School Journal, 109(4), 1–22.
Grabensberger, E., Freudenthaler, H. H., & Specht, W. (2008). Bildungsstandards: Testungen und Ergebnisrückmeldungen auf der achten Schulstufe aus der Sicht der Praxis. Graz: Bifie.
Groß Ophoff, J., Koch, U., Helmke, A., & Hosenfeld, I. (2006). Vergleichsarbeiten für die Grundschule–und was diese daraus machen (können). Journal für Schulentwicklung, 10(4), 7–12.
Gustafsson and Myrberg (in prep). School inspections of Swedish schools: a critical reflection on intended effects, causal mechanisms and methods. http://schoolinspections.eu/wp-content/uploads/downloads/2012/05/Sweden_PT.pdf. Accessed November 2011.
Hanushek, E. A., & Raymond, M. E. (2002). Lessons about the design of state accountability systems. Paper prepared for ‘Taking Account of Accountability: Assessing Policy and Politics’, Harvard University.
Hattie, J. (2009). Visible learning: a synthesis of over 800 meta-analysis relating to achievement. Milton Park: Routledge.
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.
Heubert, J. P., & Hauser, R. M. (Eds.). (1999). High stakes: testing for tracking, promotion, and graduation. Washington: National Academy Press.
Hosenfeld, I., Groß Ophoff, J., & Koch, U. (2007). Vergleichsarbeiten in Klassenstufe 3 (VERA 3). Präsentation bei der 7. Tagung “Empiriegestützte Schulentwicklung” in Mainz. Landau: Universität Koblenz-Landau.
House, E., & Howe, K. (2000). Deliberative democratic evaluation. New Directions in Evaluation, 85, 3–26.
Hoxby, C. (Ed.). (2003). The economics of school choice. Chicago: University of Chicago Press.
Hughes, G., Mears, R., & Winch, C. (1997). An inspector calls? Regulation and accountability in three public services. Policy and Politics, 25(3), 299–314.
Husfeldt, V. (2011). Wirkungen und Wirksamkeit der exzernen Schulevaluation. Überblick zum Stand der Forschung. Zeitschrift für Erziehungswissenschaft. doi:10.1007/s11618-011-0204-5.
Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979). Consequences of individual feedback on behaviour in organizations. Journal of Applied Psychology, 64(4), 349–371.
Karsten, S., & Visscher, A. J. (2001). Ervaringen met het openbaar maken van schoolprestaties in Engeland en Frankrijk. In A. B. Dijkstra, S. Karsten, R. Veenstra, & A. J. Visscher (Eds.), Het oog der natie: scholen op rapport (pp. 36–53). Assen: Koninklijke Van Gorcum.
Kerr, S. (1975). On the folly of rewarding A, while hopping for B. The Academy of Management Journal, 18(4), 769–783.
Klerks, M. (submitted). The effect of school inspections: a systematic review. Oxford Review.
Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance. Psychological Bulletin, 119(2), 254–284.
Kogan, M., & Maden, M. (1999). An evaluation of evaluators: the OFSTED system of school inspection. In C. Cullingford (Ed.), An inspector calls; Ofsted and its effect on school standards (pp. 9–32). London: Kogan Page Limited.
Koretz, D. M. (2003). Using multiple measures to address perverse incentives and score inflation. Educational Measurement, 22(2), 18–26.
Kotthoff, H. G., & Böttcher, W. (2010). Neue Formen der „Schulinspektion“: Wirkungshoffnungen und Wirksamkeit im Spiegel empirischer Bildungsforschung. In H. Altrichter, K. Maag Merki (Eds.), Handbuch Neue Steuerung in Schulsystem (pp. 295–325). Wiesbaden: VS Verlag für Sozialwissenschaften.
Kuper, H. (2005). Evaluation im Bildungssystem. Stuttgart: Kohlhammer.
Leeuw, F. L. (2003). Reconstructing program theories: methods available and problems to be solved. American Journal of Evaluation, 24(1), 5–20.
Leutner, D., Fleischer, J., Spoden, C., & Wirth, J. (2007). Schulrückmeldungen in landesweiten Lernstandserhebungen. Präsentation bei der 7. Tagung “Empiriegestützte Schulentwicklung” in Mainz. Landau: Universität Koblenz-Landau.
Luginbuhl, R., Webbink, D., & De Wolf, I. (2009). Do inspections improve primary school performance? Educational Evaluation and Policy Analysis, 31(3), 221–237.
Maier, U. (2006). Können Vergleichsarbeiten einen Beitrag zur Schulentwicklung leisten? Journal für Schulentwicklung, 10(4), 20–28.
Maier, U. (2007). Lehrereinschätzungen zu zentralen Tests und Leistungsrückmeldungen. Präsentation auf der 7. Tagung “Empiriegestützte Schulentwicklung” in Mainz. Landau: Universität Koblenz-Landau.
Malen, B. (1999). On rewards, punishments, and possibilities: teacher compensation as an instrument for education reform. Journal of Personnel Evaluation in Education, 12(4), 387–394.
Mason, R. O., & Mitroff, I. I. (1981). Challenging strategic planning assumptions; theory, cases and techniques. New York: Wiley.
Matthews, P., & Sammons, P. (2004). Improvement through Inspection. London: Ofsted.
McGarvey, N., & Stoker, G. (1999). Intervention, inspection, regulation and accountability in local government. DETR—Interim literature review. London: DETR.
McNamara, G., & O’Hara, J. (2008). The importance of the concept of self-evaluation in the changing landscape of education policy. Studies in Educational Evaluation, 34, 173–179.
McNamara, G., & O’Hara, J. (2009). Where global meets local: contexts, constraints and consensus in school evaluation in Ireland. Sage international handbook of educational evaluation (pp. 273–291). London: Sage Publications.
Nevo, D. (2006). In S. J. Greene & M. Mark (Eds.), Evaluation in education. Sage handbook of evaluation I (pp. 441–460). London: Sage Publications.
Nichols, S. L., Glass, G. V., & Berliner, D. C. (2006). High-stakes testing and student achievement: does accountability pressure increase student learning? Education Policy Analysis Archives, 14(1), Retrieved 14 November 2008 from http://epaa.asu.edu/epaa/v14n1.
Ofsted (2009). Parents’ perceptions of Ofsted’s work—a report of the NFER for Ofsted. London: Ofsted.
Peek, R. (2004). Qualitätsuntersuchung an Schulen zum Unterricht in Mathematik (QuaSUM). Empirische Pädagogik, 18, 82–114.
Peek, R. (2006). Dateninduzierte Schulentwicklung. In H. Buchen & H.-G. Rolff (Eds.), Professionswissen Schulleitung (pp. 1343–1366). Weinheim: Beltz.
Reback, R. (2005). Supply and demand in a public school choice program. New York: National Center for the Study of Privatization in Education.
Rosenthal, L. (2004). Do school inspections improve school quality? Economics of Education Review, 23(2), 143–151.
Scheerens, J. (2009). Review and meta-analyses of school and teaching effectiveness. The Netherlands/department of Educational Organization and Management. www.iqb.hu-berlin.de/lehre/dateien/rapportScherens.pdf. Accessed November 2011.
Scheerens, J., Seidel, T., Witziers, B., Hendriks, M., & Doornekamp, G. (2005). Positioning and validating the supervision framework. Positioning the supervision frameworks for primary and secondary education of the Dutch Educational Inspectorate in current educational discourse and validating core indicators against the knowledge base of educational effectiveness research. Enschede/Kiel: University of Twente/IPN
Schildkamp, K., Visscher, A., & Luyten, H. (2009). The effects of the use of a school self-evaluation instrument. School Effectiveness and School Improvement, 20(1), 69–88.
Schwippert, K. (2004). Leistungsrückmeldungen an Grundschulen im Rahmen der internationalen Grundschul-Lese-Untersuchung (IGLU). Empirische Pädagogik, 18, 62–81.
Stecher, B. M. (2002). Consequences of large-scale, high-stakes testing on school and classroom practices. Tests and their use in test-based accountability systems. In L. S. Hamilton, B. M., Stecher, S. P. Klein (Eds.), Making sense of Test-based accountability in education. Santa Monica: Rand cooperation. http://www.rand.org/pubs/monograph_reports/MR1554/.
Steffens, U. (2009). Lernstandserhebungen in den deutschen Ländern–Probleme und Perspektiven. Unv. Ms. Wiesbaden: Institut für Qualitätsentwicklung.
Toulmin, S. (1958). The uses of argument. Cambridge: University Press.
Toulmin, S. (1964). The uses of argument. Cambridge: University Press.
van Ackeren, I. (2003). Evaluation, Rückmeldung und Schulentwicklung. Münster: Waxmann.
van Bruggen, J. C. (2010). Inspectorates of education in Europe; Some comparative remarks about their tasks and work. SICI report: www.sici-inspectorates.org.
van de Grift, W. (2007). Quality of teaching in four European countries: a review of the literature and application of an assessment instrument. Educational Research, 49(2), 127–152.
Visscher, A. J., & Coe, R. (2002). School improvement through performance feedback (27–39). Lisse: Swets & Zeitlinger.
Visscher, A. J., & Coe, R. (2003). School performance feedback systems: conceptualisation, analysis, and reflection. School Effectiveness and School Improvement, 14(3), 321–349.
Whitby, K. (2010). School inspection: recent experiences in high performing education systems; literature review. Reading: CfBT Education Trust.
Wood, E. (2003). The power of pupil perspectives in evidence-based practice: the case of gender and underachievement. Research Papers in Education, 18(4), 365–383.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
Rights and permissions
About this article
Cite this article
Ehren, M.C.M., Altrichter, H., McNamara, G. et al. Impact of school inspections on improvement of schools—describing assumptions on causal mechanisms in six European countries. Educ Asse Eval Acc 25, 3–43 (2013). https://doi.org/10.1007/s11092-012-9156-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11092-012-9156-4