Advertisement

Impact of school inspections on improvement of schools—describing assumptions on causal mechanisms in six European countries

  • M. C. M. Ehren
  • H. Altrichter
  • G. McNamara
  • J. O’Hara
Article

Abstract

School inspection is used by most European education systems as a major instrument for controlling and promoting the quality of schools. Surprisingly, there is little research knowledge about how school inspections drive the improvement of schools and which types of approaches are most effective and cause the least unintended consequences. The study presented in this paper uses interviews with inspection officials and a document analysis to reconstruct the “program theories” (i.e. the assumptions on causal mechanisms, linking school inspections to their intended outcomes of improved teaching and learning) of Inspectorates of Education in six European countries. The results section of the paper starts with a summary of the commonalities and differences of these six national inspection models with respect to standards and thresholds used, to types of feedback and reporting, and to the sanctions, rewards and interventions applied to motivate schools to improve. Next, the intermediate processes through which these inspection models are expected to promote good education (e.g. through actions of stakeholders) are explained. In the concluding section, these assumptions are critically discussed in the light of research knowledge.

Keywords

School inspections Accountability External evaluation Comparative research School improvement School effectiveness 

References

  1. Altrichter, H. (2010). Schul- und Unterrichtsentwicklung durch Datenrückmeldung. In H. Altrichter & K. Maag Merki (Eds.), Handbuch Neue Steuerung im Schulsystem (pp. S. 219–S. 254). Wiesbaden: VS Verlag.CrossRefGoogle Scholar
  2. Altrichter, H., & Maag Merki, K. (Eds.). (2010). Handbuch Neue Steuerung im Schulsystem (pp. S. 219–S. 254). Wiesbaden: VS Verlag.Google Scholar
  3. Altrichter, H., & Rürup, M. (2010). Schulautonomie und die Folgen. In H. Altrichter & K. Maag Merki (Eds.), Handbuch neue Steuerung im Schulwesen (pp. 111–144). Wiesbaden: VS.CrossRefGoogle Scholar
  4. Arnold, K.-H. (2007). Bildungspolitische, diagnostische und didaktische Bedingungen und Wirkungen von Schulleistungsevaluationen. Empirische Pädagogik, 21, 448–457.Google Scholar
  5. Belfield, C. R., & Levin, H. N. (2009). Market reforms in education. In G. Sykes, B. Schneider, D. N. Plank, & T. G. Ford (Eds.), Handbook of education policy research (pp. 513–527). New York: Routledge.Google Scholar
  6. Bonsen, M., & Gathen, J. (2004). Schulentwicklung und Testdaten. In H. G. Holtappels, K. Klemm, H. Pfeiffer, H.-G. Rolff, & R. Schulz-Zander (Eds.), Jahrbuch der Schulentwicklung. Band 13 (pp. S. 225–S. 252). Weinheim: Juventa.Google Scholar
  7. Brimblecombe, N., Shaw, M., & Ormston, M. (1996). Teachers’ intention to change practice as a result of OFSTED school inspections. Educational Management & Administration, 24(4), 339–354.CrossRefGoogle Scholar
  8. Brunsden, V., Davies, M., & Shevlin, M. (2006). Anxiety and stress in educational professionals in relation to Ofsted. Education Today, 56(1), 24–31.Google Scholar
  9. Buckley, J., & Schneider, M. (2003). Making the grade: comparing DC charter schools to other DC public schools. Educational Evaluation and Policy Analysis, 25, 203–216.CrossRefGoogle Scholar
  10. Chapman, C. (2001). Changing classrooms through inspections. School Leadership Management, 1(1), 59–73.CrossRefGoogle Scholar
  11. Coe, R. (2002). Evidence on the role and impact of performance feedback in schools. In A. J. Visscher & R. Coe (Eds.), School improvement through performance feedback (pp. S. 3–S. 26). Lisse: Swets & Zeitlinger.Google Scholar
  12. Cook-Sather, A. (2002). Authorizing student’s perspectives: towards trust. Dialogue and Change in Education Educational Researcher, 19(5), 2–14.Google Scholar
  13. Darling-Hammond, L. (2004). Standards, accountability and school reform. Teachers College Record, 106(6), 1047–1085.CrossRefGoogle Scholar
  14. De Wolf, I. F., & Janssens, F. J. G. (2007). Effects and side effects of inspections and accountability in education: an overview of empirical studies. Oxford Review of Education, 33(3), 379–396.CrossRefGoogle Scholar
  15. Dillon, S. (2011). The way I see it is…Whole-school evaluation in Irish post-primary schools from the perspectives of principals, teachers, parents and students. Unpublished Doctoral Dissertation. [Online] www.dcu.ie/doras.
  16. DiMaggio, P. J., & Powell, W. W. (1983). The iron cage revisited: institutional isomorphism and collective rationality in organizational fields. American Sociological Review, 48(1), 147–160.CrossRefGoogle Scholar
  17. Doolaard, S., & Karstanje, P. (2001). Gebruik van publieke prestatie-indicatoren voor schoolverbetering. In A. B. Dijkstra, S. Karsten, R. Veenstra, & A. J. Visscher (Eds.), Het oog der natie: scholen op rapport; standaarden voor de publicatie van schoolprestaties (pp. 155–173). Assen: Koninklijke Van Gorcum BV.Google Scholar
  18. Dronkers, J., & Veenstra, R. (2001). Schoolprestatie-indicatoren in het voortgezet onderwijs: start, reacties en vervolg. In A. B. Dijkstra, S. Karsten, R. Veenstra, & A. J. Visscher (Eds.), Het oog der natie: scholen op rapport; standaarden voor de publicatie van schoolprestaties (pp. 21–36). Assen: Koninklijke Van Gorcum BV.Google Scholar
  19. Dubs, R. (2006). Bildungsstandards: Das Problem der schulpraktischen Umsetzung. Netzwerk–Die Zeitschrift für Wirtschaftsbildung, 1, 18–29.Google Scholar
  20. Ehren, M. C. M., & Visscher, A. J. (2006). Towards a theory on the impact of school inspections. British Journal of Educational Studies, 54(1), 51–72.CrossRefGoogle Scholar
  21. Ehren, M. C. M., & Visscher, A. J. (2008). The relationship between school inspections, school characteristics and school improvement. British Journal of Educational Studies, 56(2), 205–227.CrossRefGoogle Scholar
  22. Ehren, M. C. M., Leeuw, F. L., & Scheerens, J. (2005). On the impact of the Dutch educational supervision act; Analyzing assumptions concerning the inspection of primary education. American Journal of Evaluation, 26(1), 60–76.CrossRefGoogle Scholar
  23. Elmore, R. F., & Fuhrman, S. H. (2001). Research finds the false assumption of accountability. Phi Delta Kappan, 67(4), 9–14.Google Scholar
  24. Eurydice (2004). Evaluation of schools providing compulsory education in Europe. http://www.eurydice.org/portal/page/portal/Eurydice. Accessed January 2011.
  25. Eurydice. (2007). School autonomy in Europe: policies and measures. Brussels: Eurydice.Google Scholar
  26. Faubert, V. (2009). School evaluation: current practices in OECD countries and a literature review. OECD Education Working Papers, No. 42, OECD Publishing. doi: 10.1787/218816547156.
  27. Gärtner, H., Füsemann, D., & Pant, H. A. (2009). Wirkungen von Schulinspektion aus Sicht betroffener Schulleitungen. Empirische Pädagogik, 23, 1–18.Google Scholar
  28. Geijsel, F., Berg, van den, R., & Sleegers, P. (1999). The innovative capacity of schools in primary education: a qualitative study. Qualitative studies in education, 12(2), 175–191.Google Scholar
  29. Geijsel, P., Sleegers, P. J. C., Stoel, R. D., & Kruger, M. L. (2009). The effect of teacher psychological and school organizational and leadership factors on teachers’ professional learning in Dutch schools. The Elementary School Journal, 109(4), 1–22.CrossRefGoogle Scholar
  30. Grabensberger, E., Freudenthaler, H. H., & Specht, W. (2008). Bildungsstandards: Testungen und Ergebnisrückmeldungen auf der achten Schulstufe aus der Sicht der Praxis. Graz: Bifie.Google Scholar
  31. Groß Ophoff, J., Koch, U., Helmke, A., & Hosenfeld, I. (2006). Vergleichsarbeiten für die Grundschule–und was diese daraus machen (können). Journal für Schulentwicklung, 10(4), 7–12.Google Scholar
  32. Gustafsson and Myrberg (in prep). School inspections of Swedish schools: a critical reflection on intended effects, causal mechanisms and methods. http://schoolinspections.eu/wp-content/uploads/downloads/2012/05/Sweden_PT.pdf. Accessed November 2011.
  33. Hanushek, E. A., & Raymond, M. E. (2002). Lessons about the design of state accountability systems. Paper prepared for ‘Taking Account of Accountability: Assessing Policy and Politics’, Harvard University.Google Scholar
  34. Hattie, J. (2009). Visible learning: a synthesis of over 800 meta-analysis relating to achievement. Milton Park: Routledge.Google Scholar
  35. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.CrossRefGoogle Scholar
  36. Heubert, J. P., & Hauser, R. M. (Eds.). (1999). High stakes: testing for tracking, promotion, and graduation. Washington: National Academy Press.Google Scholar
  37. Hosenfeld, I., Groß Ophoff, J., & Koch, U. (2007). Vergleichsarbeiten in Klassenstufe 3 (VERA 3). Präsentation bei der 7. Tagung “Empiriegestützte Schulentwicklung” in Mainz. Landau: Universität Koblenz-Landau.Google Scholar
  38. House, E., & Howe, K. (2000). Deliberative democratic evaluation. New Directions in Evaluation, 85, 3–26.CrossRefGoogle Scholar
  39. Hoxby, C. (Ed.). (2003). The economics of school choice. Chicago: University of Chicago Press.Google Scholar
  40. Hughes, G., Mears, R., & Winch, C. (1997). An inspector calls? Regulation and accountability in three public services. Policy and Politics, 25(3), 299–314.CrossRefGoogle Scholar
  41. Husfeldt, V. (2011). Wirkungen und Wirksamkeit der exzernen Schulevaluation. Überblick zum Stand der Forschung. Zeitschrift für Erziehungswissenschaft. doi: 10.1007/s11618-011-0204-5.
  42. Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979). Consequences of individual feedback on behaviour in organizations. Journal of Applied Psychology, 64(4), 349–371.CrossRefGoogle Scholar
  43. Karsten, S., & Visscher, A. J. (2001). Ervaringen met het openbaar maken van schoolprestaties in Engeland en Frankrijk. In A. B. Dijkstra, S. Karsten, R. Veenstra, & A. J. Visscher (Eds.), Het oog der natie: scholen op rapport (pp. 36–53). Assen: Koninklijke Van Gorcum.Google Scholar
  44. Kerr, S. (1975). On the folly of rewarding A, while hopping for B. The Academy of Management Journal, 18(4), 769–783.CrossRefGoogle Scholar
  45. Klerks, M. (submitted). The effect of school inspections: a systematic review. Oxford Review.Google Scholar
  46. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance. Psychological Bulletin, 119(2), 254–284.CrossRefGoogle Scholar
  47. Kogan, M., & Maden, M. (1999). An evaluation of evaluators: the OFSTED system of school inspection. In C. Cullingford (Ed.), An inspector calls; Ofsted and its effect on school standards (pp. 9–32). London: Kogan Page Limited.Google Scholar
  48. Koretz, D. M. (2003). Using multiple measures to address perverse incentives and score inflation. Educational Measurement, 22(2), 18–26.CrossRefGoogle Scholar
  49. Kotthoff, H. G., & Böttcher, W. (2010). Neue Formen der „Schulinspektion“: Wirkungshoffnungen und Wirksamkeit im Spiegel empirischer Bildungsforschung. In H. Altrichter, K. Maag Merki (Eds.), Handbuch Neue Steuerung in Schulsystem (pp. 295–325). Wiesbaden: VS Verlag für Sozialwissenschaften.Google Scholar
  50. Kuper, H. (2005). Evaluation im Bildungssystem. Stuttgart: Kohlhammer.Google Scholar
  51. Leeuw, F. L. (2003). Reconstructing program theories: methods available and problems to be solved. American Journal of Evaluation, 24(1), 5–20.Google Scholar
  52. Leutner, D., Fleischer, J., Spoden, C., & Wirth, J. (2007). Schulrückmeldungen in landesweiten Lernstandserhebungen. Präsentation bei der 7. Tagung “Empiriegestützte Schulentwicklung” in Mainz. Landau: Universität Koblenz-Landau.Google Scholar
  53. Luginbuhl, R., Webbink, D., & De Wolf, I. (2009). Do inspections improve primary school performance? Educational Evaluation and Policy Analysis, 31(3), 221–237.Google Scholar
  54. Maier, U. (2006). Können Vergleichsarbeiten einen Beitrag zur Schulentwicklung leisten? Journal für Schulentwicklung, 10(4), 20–28.Google Scholar
  55. Maier, U. (2007). Lehrereinschätzungen zu zentralen Tests und Leistungsrückmeldungen. Präsentation auf der 7. Tagung “Empiriegestützte Schulentwicklung” in Mainz. Landau: Universität Koblenz-Landau.Google Scholar
  56. Malen, B. (1999). On rewards, punishments, and possibilities: teacher compensation as an instrument for education reform. Journal of Personnel Evaluation in Education, 12(4), 387–394.CrossRefGoogle Scholar
  57. Mason, R. O., & Mitroff, I. I. (1981). Challenging strategic planning assumptions; theory, cases and techniques. New York: Wiley.Google Scholar
  58. Matthews, P., & Sammons, P. (2004). Improvement through Inspection. London: Ofsted.Google Scholar
  59. McGarvey, N., & Stoker, G. (1999). Intervention, inspection, regulation and accountability in local government. DETR—Interim literature review. London: DETR.Google Scholar
  60. McNamara, G., & O’Hara, J. (2008). The importance of the concept of self-evaluation in the changing landscape of education policy. Studies in Educational Evaluation, 34, 173–179.CrossRefGoogle Scholar
  61. McNamara, G., & O’Hara, J. (2009). Where global meets local: contexts, constraints and consensus in school evaluation in Ireland. Sage international handbook of educational evaluation (pp. 273–291). London: Sage Publications.CrossRefGoogle Scholar
  62. Nevo, D. (2006). In S. J. Greene & M. Mark (Eds.), Evaluation in education. Sage handbook of evaluation I (pp. 441–460). London: Sage Publications.Google Scholar
  63. Nichols, S. L., Glass, G. V., & Berliner, D. C. (2006). High-stakes testing and student achievement: does accountability pressure increase student learning? Education Policy Analysis Archives, 14(1), Retrieved 14 November 2008 from http://epaa.asu.edu/epaa/v14n1.
  64. Ofsted (2009). Parents’ perceptions of Ofsted’s worka report of the NFER for Ofsted. London: Ofsted.Google Scholar
  65. Peek, R. (2004). Qualitätsuntersuchung an Schulen zum Unterricht in Mathematik (QuaSUM). Empirische Pädagogik, 18, 82–114.Google Scholar
  66. Peek, R. (2006). Dateninduzierte Schulentwicklung. In H. Buchen & H.-G. Rolff (Eds.), Professionswissen Schulleitung (pp. 1343–1366). Weinheim: Beltz.Google Scholar
  67. Reback, R. (2005). Supply and demand in a public school choice program. New York: National Center for the Study of Privatization in Education.Google Scholar
  68. Rosenthal, L. (2004). Do school inspections improve school quality? Economics of Education Review, 23(2), 143–151.CrossRefGoogle Scholar
  69. Scheerens, J. (2009). Review and meta-analyses of school and teaching effectiveness. The Netherlands/department of Educational Organization and Management. www.iqb.hu-berlin.de/lehre/dateien/rapportScherens.pdf. Accessed November 2011.
  70. Scheerens, J., Seidel, T., Witziers, B., Hendriks, M., & Doornekamp, G. (2005). Positioning and validating the supervision framework. Positioning the supervision frameworks for primary and secondary education of the Dutch Educational Inspectorate in current educational discourse and validating core indicators against the knowledge base of educational effectiveness research. Enschede/Kiel: University of Twente/IPNGoogle Scholar
  71. Schildkamp, K., Visscher, A., & Luyten, H. (2009). The effects of the use of a school self-evaluation instrument. School Effectiveness and School Improvement, 20(1), 69–88.CrossRefGoogle Scholar
  72. Schwippert, K. (2004). Leistungsrückmeldungen an Grundschulen im Rahmen der internationalen Grundschul-Lese-Untersuchung (IGLU). Empirische Pädagogik, 18, 62–81.Google Scholar
  73. Stecher, B. M. (2002). Consequences of large-scale, high-stakes testing on school and classroom practices. Tests and their use in test-based accountability systems. In L. S. Hamilton, B. M., Stecher, S. P. Klein (Eds.), Making sense of Test-based accountability in education. Santa Monica: Rand cooperation. http://www.rand.org/pubs/monograph_reports/MR1554/.
  74. Steffens, U. (2009). Lernstandserhebungen in den deutschen Ländern–Probleme und Perspektiven. Unv. Ms. Wiesbaden: Institut für Qualitätsentwicklung.Google Scholar
  75. Toulmin, S. (1958). The uses of argument. Cambridge: University Press.Google Scholar
  76. Toulmin, S. (1964). The uses of argument. Cambridge: University Press.Google Scholar
  77. van Ackeren, I. (2003). Evaluation, Rückmeldung und Schulentwicklung. Münster: Waxmann.Google Scholar
  78. van Bruggen, J. C. (2010). Inspectorates of education in Europe; Some comparative remarks about their tasks and work. SICI report: www.sici-inspectorates.org.
  79. van de Grift, W. (2007). Quality of teaching in four European countries: a review of the literature and application of an assessment instrument. Educational Research, 49(2), 127–152.CrossRefGoogle Scholar
  80. Visscher, A. J., & Coe, R. (2002). School improvement through performance feedback (27–39). Lisse: Swets & Zeitlinger.Google Scholar
  81. Visscher, A. J., & Coe, R. (2003). School performance feedback systems: conceptualisation, analysis, and reflection. School Effectiveness and School Improvement, 14(3), 321–349.CrossRefGoogle Scholar
  82. Whitby, K. (2010). School inspection: recent experiences in high performing education systems; literature review. Reading: CfBT Education Trust.Google Scholar
  83. Wood, E. (2003). The power of pupil perspectives in evidence-based practice: the case of gender and underachievement. Research Papers in Education, 18(4), 365–383.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  • M. C. M. Ehren
    • 1
  • H. Altrichter
    • 2
  • G. McNamara
    • 3
  • J. O’Hara
    • 3
  1. 1.Institute of EducationLondonUK
  2. 2.Johannes Kepler Universität LinzLinz-Auhof ÖsterreichAustria
  3. 3.Dublin City University, School of Education StudiesDublin 9Ireland

Personalised recommendations