Introduction

Education systems across the world have seen many reforms over the years and changes in the modalities of governance over time. These developments vary across countries, but common threads are, according to Au and Ferrare (2015), a shift from central government towards more decentralised governance, where responsibilities for governing are increasingly taken up by public-private partnerships, appointed managers and other bodies comprised of state and corporate leaders instead of by elected state bodies. The government still has a role in governance, according to Joseph (2010), but primarily through producing the legislation and regulatory framework which define ‘a broader configuration of state and key elements in civil society’ (p. 5). Theisens et al (2016) similarly talk about a rise of New Public Governance, which followed an era predominated by New Public Management until approximately the year 2000. New Public Governance is based on horizontally organized systems with multiple centres of power which collaborate through networks. Governments are either actors in these networks, or they steer through networks by creating the arena in which networks operate, such as through establishing frameworks for collaboration, or facilitating knowledge exchange. Systems with many centres of decision-making in which the state is not the sole locus of authority, but where state and non-state actors are both regulators and regulated in a set of highly complex and interdependent relations are called ‘polycentric’ (Ostrom et al. 1961; Black 2008), as opposed to monocentric forms of steering, where the national government is the central actor in defining and designing civil society and in deciding on how to tackle societal issues through instructions, norms, policy guidelines, monitoring and control (Teisman 1992). In a polycentric regime, networks of schools and their stakeholders take a prominent role in defining, regulating and shaping school quality. Steering through networks is expected to create conditions for responsiveness which allow and motivate schools to learn from each other, to find ways to effectively develop and implement solutions to local problems and to have the capacity to respond to changing circumstances.

Examples of polycentric systems can be found throughout Europe but here we will talk about three examples from England, the Netherlands and Northern Ireland. In these countries we find cases of governments steering through networks (England and The Netherlands), whereas in West Belfast local government and district inspectors are active partners in a network of pre-primary, primary and secondary schools to improve learning outcomes of children in a historically disadvantaged area.

These moves towards a more polycentric education system have consequences for Inspectorates of Education. As schools have a role in defining and shaping educational quality in new local arrangements involving networks of stakeholders, Inspectorates of Education will be required to adapt their inspection methods. This will involve a redefinition of roles away from centralized approaches to quality control, to more agile and contextual methods of evaluation. Such methods of evaluation are needed as networking can come with a range of problems that Inspectorates of Education need to address. Common problems have been described by Mayne and Rieper (2003) and Janssens and Ehren (2016) who talk about a diffusion of roles and responsibilities with limited clarity for parents or teachers over where to complain or who to approach when things go wrong, competition between partners in the network, high transaction costs of collaboration, or convergence toward groupthink. Examples of these unintended consequences for school networks were described by Ehren and Perryman (submitted) who talk about how large Multi-Academy Trusts have introduced multiple layers of management to coordinate, top slicing schools’ budgets to finance these layers, or where collaboration between schools located in different parts of the country takes up substantial time of head teachers to travel to meetings.

This paper introduces a conceptual framework, describing a continuum of inspection models which fit a monocentric system of strong centralized steering by the state (government) to a polycentric system of steering through/within networks (governance). We will use Christie and Alkin’s (2013) work to inform our conceptual framework, and compare and contrast the inspection models in the three aforementioned systems (England, the Netherlands, Northern Ireland) to this framework, presenting emerging good practices and discussing the constraints and dilemmas the Inspectorates in these three systems are facing as they move towards fully polycentric models. First we will describe the types of networks Inspectorates of Education in our study engage in before we conceptualize ideal-typical models of inspections in a polycentric context and present examples from the three systems.

Educational networks

Networks are defined here as formal arrangements of a multitude of public and private organizations, agencies, and departments that have been constituted to facilitate collective action (see Provan and Kenis 2008). Informal networks where schools collaborate without structured agreements on collective action (‘soft collaboration’) are outside of the scope of this paper.

Collective action can include the implementation of specific education reforms or services, joint professional development, school improvement and exchange of good practices and/or peer reviews. These collective actions imply that (aspects of) the provision and/or improvement of teaching and learning becomes the joint responsibility of the network, instead of that of single schools. Of course in different countries the degree of autonomy granted to individual schools or networks varies a great deal, as do the formal structures underpinning collective actions. Schools ‘have been given decision rights of different extent and in different fields at very different points in time’ (Altrichter et al. 2014, p. 3). Nonetheless a significant degree of commonality applies. Typically education networks would include schools and their governing bodies (within or across different schooling phases), and potentially also other service providers such as youth services or local community workers. These networks are often underpinned by legal structures which formalise the relationships between these institutions. In some cases, a separate governing body is added to the network to coordinate the partnership work and provide support services to individual schools, sometimes even taking over some of the responsibilities and leadership from/of individual schools.

These networks (and their governing bodies) can be the object of inspections when the Inspectorate of Education evaluates the quality of the collaboration between partners in the network or outcomes generated by the collaborative efforts of the network. Such an approach would fit the previously described approach of steering through networks, where central government sets frameworks, formulates a vision, facilitates knowledge exchange, or acts as a crowbar to enhance collaboration. In a polycentric system, where governments become actors in the network, the Inspectorate would however typically also be part of the network and be involved in the networking process. In the following sections, we will use Christie and Alkin’s framework of evaluation theories (2013) to explain the shift in inspection methodology required, when the network becomes the object of, as well as the platform for the implementation of evaluations.

Conceptualizing inspections in a polycentric system

Inspections are a widely used form of school evaluation, according to Christie and Alkin (2013). Their tradition of designing and employing evaluation procedures whereby teams with presumed expertise, guided by established process standards, visit a site to observe, account, make a report and judge institutions plays a significant role in evaluation. Janssens and Dijkstra (2013) previously used Alkin’s (2013) framework of evaluation theories to position inspections of individual schools, describing how evaluation theories inform inspection methodology, inspection judgements and the user focus of the evaluation effort. In this section their thinking is extended by using Alkin’s framework to reflect on the changes in inspection methodologies, judgements and user focus when Inspectorates of Education change their object of evaluation from individual school quality to the functioning of networks of schools and/or stakeholders.

Methodology

Methodology concerns the techniques used to conduct evaluation studies and these can range from the traditional research methods of (quasi) experimental research to evaluate the effects of an intervention or programme, to broader and more comprehensive conceptualizations of evaluation of human activity, policies or organizations. Alkin (2013) provides a summary of the main (North American) theorists that have developed evaluation methodologies, such as Campbell, Suchman, Boruch, Cook, Cronbach, Rossi, Weiss, Chen, Henry, Mark and Tyler. His overview of the main ideas of these evaluation theorists suggests a continuum of approaches from, on the one hand, those which are concerned with

  • investigating causal inference and making generalizations to other subjects and settings,

  • evaluating single interventions, programmes, or organisational entities, and

  • using objectivist and standardized techniques in the evaluation,

to those which aim to capture

  • the mechanisms and conditions that explain the functioning and performance of an intervention, programme or organisation. Such approaches would,

  • include multiple levels of analysis (individual, interpersonal and collective) at which influence occurs, and

  • using constructivist approaches to develop and test theories of ‘how something works’.

The first approach typically fits within monocentric systems where Inspectorates of Education would use standardized frameworks, aiming to produce evaluation findings that are reproducible and that would lead to similar assessments of school quality across different inspectors (Janssens and Dijkstra 2013). Such objectivist and standardized approaches are informed by education policy frameworks that describe school quality and define the remit of Inspectorates of Education. They rely to a large extent on available quantitative data, such as students’ test scores or student drop out rates to evaluate the school’s performance, sometimes using risk-based approaches which assume a cause and effect relation between risks of failing school quality (e.g. staff turnover, low teaching quality) and student performance. Of course inspection even in monocentric environments focusing largely on individual schools varies hugely from system to system and those with less emphasis on high stakes accountability and more on development and improvement fit more comfortably into the second approach above.

By definition a polycentric context implies that the power and control over who defines and monitors school quality is more fragmented. Interactions about school quality do not stop at the borders of an individual school but are shaped in interdependencies of schools and their stakeholders who have different roles and expertise in defining and improving school quality. The second set of approaches becomes more relevant for Inspectorates of Education that have to adapt to local context and the type of networks they are inspecting and to create the conditions in which such networks effectively steer themselves. The common value of these approaches is their ability to understand and validate local and context-specific approaches to shaping educational quality by different partners in a network, looking at the bigger picture of how the many different parts in a network operate and the ways they interact and evolve over time in mutually reinforcing ways.

Valuing/judging

Valuing and judging distinguish inspection from other forms of evaluation or research as inspectors must place value on their findings and often have to make judgements about the quality of some object, situation or process. Valuing and judging are an important part of Inspectorates of Education’s work which is often structured by a set of clear protocols and guidelines to judge the quality of individual schools. Many inspection systems categorize schools on a four-point scale (ranging from failing to good), using a hierarchical model of aggregating judgements on lower level indicators to a summary score on the overall quality of the school.

These ‘monocentric’ approaches are strongly in line with the objectivist methodologies described in the previous section, putting the onus on the judgement of the inspector who is evaluating a school, making ‘pass/fail’ decisions or using standardized evaluation criteria to compare similar entities, or benchmarking schools against a set of inspection indicators. Such a standardized ‘objectivist’ approach however does not fit well in a polycentric system where a variety of different networks emerge which include a range of (sometimes different and changing) actors working on a variety of different network-level outcomes in response to context-specific problems.

Christie and Alkin (2013) describe more ‘subjectivist’ approaches to valuing which are responsive to the object of evaluation and guided by the meanings people construct. Stakeholders and users of an evaluation (such as the network that is inspected) are actively involved in making judgements as the evaluator ensures that multiple realities are taken into account when making a value judgement. Relevant methodological approaches as described by Stufflebeam, Wholey, Chelimsky, Alkin, Patton, Fetterman, Cousins, Preskill, King (see Christie and Alkin 2013) include the ‘context, input, process and product evaluation model’, Wholey’s four-stage procedure for sequential purchase of information, ‘utilization-focused evaluation’, ‘developmental evaluation, ‘empowerment evaluation’ (building on self-evaluations of users), ‘participatory evaluation’, and ‘interactive evaluation practice’. The common concepts underlying these approaches are:

  • a focus on the process of evaluation and a continuous cycle of evaluation with the purpose of transformation and learning (instead of seeing evaluation as an end product to be used for improvement by stakeholders)

  • involvement of stakeholders throughout the evaluation process (instead of treating them as end users), and

  • a shift in the role of evaluators from objective outsiders to one which fosters continuous interaction with the major stakeholders in an evaluation; evaluators should be actively involved in developing intended users’ commitment to utilization of ideas for improvement.

Such a shift in the position of schools, their stakeholders and the Inspectorates of Education who now become equal partners in a more interactive and ongoing evaluation of education quality also implies a different set of consequences to motivate improvement. Instead of using sanctions, rewards and interventions in single schools, Inspectorates of Education now need to develop a set of intelligent strategies that would enhance the performance of the entire network. This might be achieved by purposefully providing relevant actors with the information to act on inspection findings, putting strategies in place to shift the power balance to improve relations in the network and increasing transparency to external stakeholders. Joint learning among all participating agencies and organisations in the network and the Inspectorate through a process of collaborative evaluation and knowledge development is the underpinning rationale for user involvement, and reciprocal relationships and joint activities become essential strategies in the evaluation and monitoring of schools.

Table 1 summarizes the two ends of the continuum of on the one hand inspection approaches in a monocentric system, using standardized frameworks to judge (single) school quality to inspections that fit in a polycentric system, using a more qualitative, interpretative and flexible approach of validating good practices of localized and collaborative provision and improvement of education.

Table 1 Inspection and evaluation approaches in a monocentric and polycentric education system
Full size table

Research methodology

A preliminary scoping exercise indicated that particularly the Netherlands, England and Northern Ireland have seen recent shifts in education systems towards more polycentric settings that have had implications for their Inspectorates of Education.

England has recently introduced a number of reforms that aim to create a self-improving system where schools collaborate in networks to exchange good practices and maximize inter-school professional development. There are a range of networks in place, such as teaching school alliances, national and local leaders of education who support groups of schools in improving, learning networks which are organized by local authority improvement officers, and other types of collaboration around peer review and improvement. Here we focus on Multi-Academy Trusts, which are the most widespread type of (formal) networks of schools. As a recent NfER report (Worth 2015) explains, Multi-Academy Trusts (MATs) are the Department of Education’s preferred governance structure for academies and national policy has seen a number of incentives to motivate schools to become part of a Trust. MATs are considered to be the best long term formal arrangement for stronger schools to support the improvement of weaker schools, and the collaboration of schools, under the authority of a Trust, is thought to improve the quality of teaching and the richness of children’s learning.

Northern Ireland has also seen a number of reforms aimed at enhancing collaborative arrangements between communities of schools in a geographical area. One such example is through the establishment of (voluntary) ‘Area Learning Communities’ (ALCs) which are clusters of schools who plan collaboratively to meet the needs of pupils in an area and to focus on sharing good practice. ALCs work together to provide a broad and balanced curriculum and to deliver on the statutory requirements of the ‘Entitlement Framework’.Footnote 1 The Entitlement Framework requires schools to provide pupils with access to a minimum number of courses at Key Stage 4 (24 courses) and a minimum number of courses at post-16 (27 courses). At present there are 30 ALCs in Northern Ireland (Department of Education Northern Ireland 2010, p. 4).

In the Netherlands, mainstream schools and special schools are, since 2014, required to work in partnerships to provide inclusive education for all children (also children with disabilities) under 76 new authorities for primary education, and 74 for secondary education. These networks were centrally formed by the Ministry of Education, Culture and Science according to their regional proximity, number of pupils, existing informal cooperation between schools, and after consultation with the school boards. Each network of mainstream and special needs schools is now governed by new education authorities who are responsible for ensuring close collaboration between these schools in the provision of care and high-quality education to each pupil. They have a legal ‘duty to care’ which means that they are formally responsible for finding an adequate school place for each pupil in their area, instead of parents who used to be responsible for the placement of their child in a school. The network authority also receives a budget to provide for in-school support of children with learning/physical disabilities; they are required to develop an action plan in which they outline how this support is organized and funded within and across schools in the network. Schools are also governed by a (separate) school board whose portfolio of schools often does not overlap with the schools in the network for inclusive education, creating two distinct collaborative arrangements for schools.

In each of the three systems, Inspectorates of Education inspect a network of schools and include indicators in their framework on how schools are working together to support school improvement and/or provide inclusive education to children with special needs and/or learning disabilities. The systems vary in the extent to which these methods and frameworks are fully developed and implemented. The three systems and inspection models are by no means perfect representations of our conceptual framework, or representative for the types of inspections of networks across Europe, but they are presented here as illustrations of our conceptual framework, allowing us to understand potential challenges and opportunities for other Inspectorates of Education who see their education systems shift towards a more polycentric structure.

The study of the three examples presented here included a documentary analysis and interviews with representatives of inspection agencies (e.g. policymakers/inspectors) and representatives from the educational network in each country to describe the shift in roles and responsibilities of their Inspectorates of Education. The respondents were selected to represent the actors within each country who are part of/in charge of a school network (using a convenient selection of an exemplary network) and those who are responsible for the accountability of those networks. The descriptions present inspection models that were in place in 2014–2015. The analysis of relevant documents and interviews transcripts was used to provide a description of the methodology, valuing and user involvement in inspections as outlined in the previous section.

Table 2 provides an overview of key documents and interviewed participants. The data was analysed and reported for each country separately, and a summary is presented in the following section.

Table 2 Overview of data collection
Full size table

Examples of inspections in a polycentric context

This section describes inspection practices in England, the Netherlands and Northern Ireland that fit our theoretical conceptualization of inspections in a polycentric context, following Table 1.

England

The formation of Multi-Academy Trusts (MATs) and the creation of a self-improving system imply a significant shift for England’s Inspectorate of Education, Ofsted. Their inspections have mostly been predicated on assessing the performance of individual institutions, and the current inspection framework is still largely focused on inspections of individual schools. Recent inspection practices have however seen an increased focus on the evaluation of the support that schools provide to other schools and the support they receive from their governing body as an element of the inspection judgement of the quality of leadership in each school. As a result, schools can now only be judged to be outstanding if they actively support other schools in their improvement and are an active partner in the network in which they operate.

The recent establishment of MATs has however also seen the introduction of a new model of ‘focused inspections’ which looks at the functioning of the Trust and the collaboration of schools in the Trust. The inspection frameworkFootnote 2 and a brief for inspectors who participate in coordinated inspections explain how such focused inspections include coordinated visits to all the schools in a Trust. Inspectors who lead on inspections of individual schools within a focused inspection will look for differences and common issues between schools in analysing questionnaires from students, parents and staff during inspection visits (Inspection framework, and HMI interview). Common issues across schools will be shared and discussed between the lead inspectors of the individual school inspection teams during the inspection week(s), particularly to agree on any features that could result in common areas for further improvement across the Trust. Focused inspections also evaluate the level of support and challenge individual schools provide each other and receive from the Trust (HMI interview and Ofsted annual report 2013/2014). There is no common framework for focused inspections and MAT reviews, as one of the interviewed HMI explains:

the focused inspections are the inspections of the individual academy that take place in week one, and they take place in the window in which an academy or a school is due to be inspected, they can’t come out of that window. So that’s what the focused inspections are, and they contribute, or form the basis of, the MAT review along with the phone calls (with head teacher of schools who are not inspected), so they are a part of the jigsaw, a big part of the jigsaw. The main part obviously as far as the MAT is concerned, and the other stuff is really gathering information on performance and the impact of the MAT’s leadership and governance on the individual academies. There is no framework for this type of review, although we do have a form of guidance but it’s not a published framework, but the approach we would use is a common sense approach.

The focused inspections result in an outcome letter which includes a summary of strengths and weaknesses of the schools across the Trust, providing an overview of the judgment of each school in the Trust. Examples of weaknesses in published outcome lettersFootnote 3 are low standards in specific subject areas across schools in specific grades, or weaknesses in middle leadership and governance. Strengths may include the support of individual academies by the Trust, the opportunities for school staff for secondments and professional development, and the impact of central human resource services in managing underperforming staff and supporting recruitment across the schools in the Trust.

These approaches are different to the highly scripted inspection methods of individual schools; the reason being that Ofsted has no legal remit to inspect Trusts and is using a specific reference in the inspection framework which allows them ‘to inspect more frequently schools that are part of a formal grouping of schools who share important aspects of its provision’ (letter of the Secretary of State, 22 January 2015, pp. 1–2Footnote 4).

The absence of a standardized framework has however, according to interviewed HMI, led to different conceptualizations of the quality of MATs; one respondent explained how the Department of Education came up with a list of high performing Trusts, which were not considered to be of good quality by Ofsted. The practice of publishing outcome letters instead of standardized inspection reports was also put in place, according to one HMI, to circumvent the lack of legal power to formerly inspect the functioning of Trusts. As a result, the focused inspections primarily evaluate the functioning of the Trust by aggregating the findings from single school inspections, looking at the collaboration between schools and the support each school has received from the Trust, without a clear set of indicators on how to evaluate network-level outcomes of the Trust and the quality of its centralized services.

Additionally, the Department of Education now employs (since 2014) Regional Schools Commissioners who make decisions on applications from schools wanting to become academies and organisations wanting to sponsor an academy. They are also responsible for the oversight and monitoring of those academies which are put in special measures by Ofsted or require improvement.Footnote 5 The National and Regional Schools Commissioner we interviewed, explain how, in cases of underperformance in an academy school or Trust, they take their authority from the funding agreement (the contract between the Secretary of State and the Trust) to monitor the performance of an academy and/or its Trust. Monitoring arrangements differ across regions but generally include Ofsted-like visits, as well as ‘soft intelligence’ which is informed by conversations of the commissioners and the local authority and regional head teacher boards.

Northern Ireland

The inspection of area-learning communities are, as with all modes of school inspection in Northern Ireland, managed by the Education and Training Inspectorate (ETI).Footnote 6 Although single school inspections are a priority for the ETI, area inspections are also used to evaluate a particular aspect of education across different stages in a geographical area.

The framework for area based inspections is similar to that of inspections of single schools in that quality indicators, areas for improvement, etc. form part of the framework. According to an inspector in Brown et al. (2015) ‘the framework in itself is just like an inspection framework but is more wide-ranging than for a school’ (p. 40). In the case of the area inspection of West Belfast, the overarching theme for the area inspection related to: ‘strategic planning for education and training within the area; the quality of learning for young people within the area; and the effectiveness of the transition arrangements for young people within and across the various sectors’ (Education and Training Inspectorate 2010, p. 4). The network sets specific objectives for each of these topics and these become the organizing point for the self-evaluation of the network and for each organisation within the network, as well as for the area inspection by the ETI.

Area inspections include a random sample of education providers in the area who are visited within a specific time frame. The ETI asks these providers to send in relevant documents in preparation for the visit, such as student attendance, student performance in external examinations, and the results of previous inspections. The ETI also requests that each organisation complete a self-evaluation report on the strengths and areas in their organisation prior to the inspection taking place.

As Brown et al. (2015) explain, during the inspection, each inspector in the team evaluates a representative sample of education providers relating to their own specialism, such as pre-school centres, primary schools, post-primary schools, alternative education providers, special schools, and further education or youth settings. Data collection (observations, interviews, analysis of examination data, minutes of meetings) during the visit would typically include a range of organisations in the area such as the Education and Library Board and the curriculum advisory support service,Footnote 7 and stakeholders in these organisations (students, parents, teachers, members of the middle and senior management team, and members of Boards of Governors). Self-evaluation reports of each of these organisations on the topic of the inspection (e.g. transition of students), as well as of the entire network are an important starting point for the inspection as it will inform the data collection and analysis during the visit.

When the inspection is complete, each organisation receives its own inspection report, detailing the quality of educational provision and areas for improvement relating to the focus of the inspection in their organisation. Additionally, an area-based report is issued which provides an overall judgement of the collective performance of the inspection area (ranging from unsatisfactory to outstanding), the main strengths and areas in need of improvement, and a quantitative description of the extent to which the objectives of the network have been reached (ranging from ‘almost/nearly’ to ‘very few/a small number’).

There are no formal consequences resulting from an area inspection. Rather, Brown et al. (2015) describe how area inspections in West Belfast are predicated on supporting stakeholders in their ongoing improvement through the promotion of rigorous self-evaluation.

Respondents from the key stakeholder, the ALC, the Inspectorate but also schools and other providers are overwhelmingly positive about the potential of this approach to inspection. In brief three key themes emerge in interviews. The first is that this type of networking enables and improves collaboration and reduces competition between organisations, facilitating initiatives such as better transition between primary and secondary schools, shared curricula in key areas of literacy and numeracy and joint staff training initiatives. The second theme is the extent to which this type of evaluation has shifted the emphasis in inspection from accountability to encouraging improvement and in particular to the use of self-evaluation based on first hand evidence to inform both school and network activities. The third theme is the way in which a network can facilitate strategic planning or ‘joined-up thinking’ in a new way but that this process is helped by external support from both the ALC and the Inspectorate.

The Netherlands

The networks for inclusive education are relatively new in the Netherlands and the Dutch Inspectorate of Education only recently developed a new framework for the inspection of these networks.Footnote 8 The coordinating inspector explains how these frameworks were developed in close cooperation with school boards and the new education authorities and includes indicators on:

  • Outcomes: the extent to which each school in the network and the network collaboratively provides adequate support to all pupils and has facilities and structures in place to provide such support.

  • Management and organization: the achievement of the network’s mission and goals within the requirements set by legislation, the internal communication and management of the network and the collaboration between schools to achieve these goals.

  • Quality assurance within the network, and its implementation of systematic self-evaluations to assess strengths and weaknesses and implement improvements.

The framework was implemented in 2015/2016 and follows a risk-based approach, which is described in the inspection framework for networks of inclusive educationFootnote 9. A comparison with the framework for primary and secondary school inspectionsFootnote 10 shows the similarity in approaches where an inspection starts with an early warning analysis of available data to understand potential risks in the functioning of these networks (e.g. looking at number of students out of school, transfer of students between schools). High-risk networks are subjected to an additional expert analysis in which school inspectors analyse the pupil referral and support policy of the network, the annual report of the network and the distribution of support services (including special needs teachers) across schools and other signals of the functioning of (schools in) the network, such as press releases or complaints from parents or other stakeholders. In case this expert analysis indicates potential risks of failing, inspectors schedule interviews with the education authority of the network or its executive manager to discuss risks; a follow-up visit of schools in the network is issued when necessary.

The coordinating inspector explains how the Inspectorate of Education publishes the outcome of the risk-analysis (which can be ‘basic’ when the initial desk research and expert analysis shows no risks), and the report from visits of the network. The report provides an assessment and overview of strengths and weaknesses on the inspection framework that education authorities are expected to address. Failing networks are, according to the coordinating inspector, subjected to increased inspection monitoring while a regional coordinator can also be appointed to take over some of the responsibilities of the network authority.

The coordinating inspector and the chair of the board of the network talk about a number of issues in the implementation of these new types of inspections. According to both respondents, there is limited alignment between the inspection of individual schools in the network, and the inspection of the network although both frameworks have similar indicators of support of children with learning difficulties. In both types of inspections, schools are judged on the quality of their support to children. However, inspectors involved in both types of inspections are working in different divisions within the Inspectorate and have little communication about their inspections of schools and the network.

The coordinator of the network and the chair of the network board also talk about a mismatch between their own internal structure and the allocation of inspections. Both respondents explain how the network, which is comprised of 165 primary mainstream schools, 6 primary special schools and 9 schools for children with severe disabilities, was split into three smaller and regionally closer subnetworks to streamline the support to children. These three small subnetworks are the main organizing entities for the provision of inclusive education, whereas the Inspectorate only looks at the functioning of the whole. This mismatch complicates the preparation for external inspections and the use of inspection findings for improvement of the network.

Discussion and conclusion

This paper used Christie and Alkin’s framework of evaluation theories (2013) to provide an overview of methodology, valuing and user involvement that Inspectorates of Education can use when moving from a traditional ‘monocentric’ top-down model of inspection and evaluation to a more lateral ‘polycentric’ one. We presented three examples of Inspectorates of Education who have developed new models to inspect networks, and in West Belfast, have become part of the network they inspect. The variety of approaches illustrates how inspections support governments in steering through networks or where governments become one of the actors within a networked education system. In England we described Ofsted’s focused inspections of Multi-Academy Trusts which evaluates the collaboration between schools in the Trust and the support from their Trust; in the Netherlands, we looked at the inspections of networks for inclusive education, whereas our Northern Ireland case described the area-based inspections in West Belfast.

Single school inspections as the main organizing principle

A comparison of the three cases suggests that these newer ‘polycentric models’ of inspections are to a large degree informed by single school inspection models that were already in place in these countries, following existing approaches for individual schools and using similar ways of collecting data. The Dutch Inspectorate of Education has for example adapted existing early warning analyses and risk-based inspections of individual schools to the inspection of the new school networks. Specific indicators and data within the two (single school, and network) models are different, but the overall inspection methodology and involvement of, and reporting to stakeholders is essentially the same. A similar conclusion can be drawn for England where focused inspections of MATs are primarily an aggregate of the outcomes of the inspections of single schools (publishing aggregated league tables to rank order MATs), and only the support of the Trust to individual schools captures outcomes on the level of the network. Northern-Ireland is the only example where inspections of networks seem to move beyond a copy of existing models of single school inspections. The area-based inspections in Northern-Ireland specifically capture network-level outcomes that are informed by a network-level self-evaluation that is implemented and quality assured by the network and their district inspector. The fact that the area-learning communities have a common goal and need to collaborate when trying to meet legislative requirements of offering a minimum of 24–27 subject areas may have offered a clear purpose to move beyond inspections of single schools. However, in Northern Ireland, the area-based inspections were recently also abolished in favor of single school inspections, suggesting that single school inspections remain the main organizing principle for Inspectorates of Education. Our case studies suggest a range of explanations as to why Inspectorates of Education are constrained in moving towards more polycentric models and some of the dilemma’s they are facing when trying to redefine existing practices:

Legislative and political position of inspection systems

The first obstacle seems to lie in the strong legislative positioning of Inspectorates of Education which limits a more flexible approach in evaluating locally relevant issues. The underlying argument is that inspections need to be transparent and set in legislation to allow them to sanction schools. The fact that frameworks, roles and responsibilities of many inspection systems are prescribed in legislation inhibits the ability of Inspectorates of Education to adapt to a variety of different contexts, particularly when there is a climate of high stakes accountability and strong political scrutiny over who is inspected, on what areas of quality and when. The legislative positioning of inspection systems also implies that accountability of school networks primarily takes into account school partnerships that have a formal and statutory basis where the collaboration and governance of the network of schools is set out in a formal agreement. Such an agreement and formal basis allows the Inspectorate to know about existing networks and have a clear line of (hierarchical) authority and governance to hold the network accountable for its performance. Inspection systems seems to be limited in evaluating more fluid and informal types of partnerships that lack a formal authority. Their judicial approach to evaluation and inspection does however not sit well with the need to develop more flexible approaches of connecting stakeholders in the system with a focus on understanding why and how specific solutions work in specific contexts, and how these feed into, or are shaped by, policy on the national level. Less strict policy frameworks are needed to create high-quality iterative and evidence-based feedback loops that would inform system-wide improvement.

Culture of hierarchy and objectivity

Inspection systems are traditionally positioned in hierarchical arrangements where individual organizations are accountable to national government. Such arrangements include inspections of single schools and other accountability arrangements (e.g. high stakes testing) where sanctions and increased monitoring are put in place for failing schools to improve performance on a centrally defined framework. A repositioning of inspection systems as a partner within a network of schools suggests a shift in power balance where schools and inspectors are now equal partners in defining and evaluating, which does not sit well with the notion of providing an objective external assessment of school quality on behalf of central government and for the public good. As Ehren and Perryman (submitted) explain, Inspectorates of Education generally feel they need to operate as objective outsiders who evaluate school quality for the purpose of the common good; close collaboration with their object of evaluation is often seen as a potential source of bias of the inspection assessment, and standardized and centralized frameworks are put in place to enhance the accuracy and comparability of the judgement while also safeguarding schools against personal preferences of individual school inspectors.

A shift towards horizontal and lateral inspection approaches also implies a greater responsibility of the network to set the agenda for evaluations and have the skills and commitment to evaluate the quality of the collaboration of schools in the network, and the contribution of each partner to network-level outcomes. Partners in the network need to collect information on these newer outcomes and need to develop indicators showing the aggregate results of emergent collaboration.

In many education systems, data (e.g. on student achievement, drop-out, pupil-teacher ratio) is organized on the school level and needs to be reorganized to represent the performance of the network. Additionally, the network also needs to develop mechanisms to act on such information, e.g. by switching ineffective transactions between partners in the network, or by excluding partners from the network who degrade overall outcomes of the network. As Kania and Kramer (2011) explain, developing shared measurement, and collecting data and measuring results consistently on a short list of indicators at the network level and across all participating organizations not only ensures that all efforts remain aligned, it also enables the participants to hold each other accountable and to learn from each other’s successes and failures.

Disconnect in accountability arrangements

Our findings also suggest that the introduction of new accountability arrangements for school networks are to some extent introduced on top of existing systems. In the Netherlands there are two separate divisions within the Inspectorate responsible for the single school inspections and inspections of networks who work with two separate frameworks; they share little knowledge about the schools and networks they inspect or the reports they write, and there is little coordination of visits and interventions. In England, the focused inspections and Ofsted’s reviews of MATs, and the monitoring of academies and MATs by Regional Schools Commissioners is added to the inspections of single schools. The regions Ofsted works in and those of the RSCs are not coterminous which complicates the coordination of their work. Only Northern Ireland sees a clear link between the inspection of individual organisations and their network through the establishment of a district inspector who acts as a liaison between the schools and the ETI, and ensures that the inspections of single schools and the network are closely connected. Such connection is needed to prevent a top heavy and multi-layered accountability system where schools (in networks) are potentially confronted with inconsistent and conflicting demands and unclear and ambiguous performance targets. As Ehren and Hatch (2013) have outlined, such systems may have unintended consequences when schools respond defensively or seek out the most expedient or obviously acceptable position, preventing them from learning and trying out new solutions.

Looking forward

Finally we ask ourselves whether polycentric inspection models can and should replace single school inspections. The question is a relevant one, given the value of single standardized school inspections in ensuring a coherent system where schools are encouraged to organize teaching and learning along similar standards and indicators of quality. The contextual nature of ‘polycentric models of inspections’ implies a more fragmented system where information on educational quality is less easy to compare and with limited opportunities for benchmarking and exchange of good practices.

The value of an evaluation of networks has however equally been recognized in the three cases in our study with the effect of adding inspections of networks to existing single school inspection systems. A question that emerges is whether such a top heavy accountability system is an efficient use of resources (our West Belfast case study suggests that it is not), and also if the two types of inspections can effectively be combined. Our conceptual framework clearly indicated the differences in approaches and underlying rationale of evaluation approaches in monocentric versus polycentric systems, particularly in the use of standardized versus contextualized methodology, the role of stakeholders and the shifting position of Inspectorates from the top of the hierarchy to being an equal partner in the evaluation and accountability of networks. The two approaches require a distinct shift in power balance, and a restructuring of roles and responsibilities of Inspectorates of Education and schools in a network which seem problematic to combine. A decentralized inspection model will only truly work in high trust education systems which have genuinely moved to a more decentralized networked structure where central government has released control to the local level, when there are clear structures in place that allow schools to fulfil their autonomy and when all the partners in a network have the capacity, the expertise and the maturity to take on this responsibility. Replacing single school inspections with more laterally organized inspections of networks is therefore only a valid option if the education system of which they are part has made a similar shift.