- First Online:
- Cite this article as:
- MacBeath, J. & Moos, L. Educ Asse Eval Acc (2009) 21: 1. doi:10.1007/s11092-009-9069-z
- 224 Views
Educational Evaluation, Assessment and Accountability. These are three key ideas that occupy the primary space in educational practice and policy making in virtually every country in the so-called developed world. Yet nowhere is there agreement as to the meaning of these highly contested terms and even less consensus as to their inter-relationship either in policy making or in school and classroom practice. There is, on the other hand, a substantial body of research and educational literature, some empirical, some speculative and much framed within its own theoretical discourse. This journal aims to enrich that discourse through leading edge research and critique in three areas so vital to the health of educational systems and the children those systems are entrusted to serve.
It [‘Outsidedness’] aims at understanding not by identification (‘they are like us’) but by the recognition of differences — ‘we are different from them and they are different from us; by exploring these differences we will understand ourselves better.’ (Czarniawska 1997, p.62)
Coming to understand ourselves better might, at first sight, seem a modest aim for an academic publication but whether as an end or a starting point it is a seminal aspiration of this journal, its editors and editorial board. One of the criteria for judgement of submissions will be the extent to which they help us step outside our immediate frames of reference and to think again.
There is of course a substantive body of research journals which already address these very issues, so that the need for yet another journal may not be immediately apparent. ‘Educational Evaluation, Assessment and Accountability’ is, however, not a totally new addition to the field as it is benefits from the legacy of ‘Personnel Evaluation’, while recasting the scope, readership and international appeal of that publication. Members of the Personnel Evaluation journal remaining with us on the editorial board provides the continuity while new members of the board — from Australia, Canada, Denmark, England, France, Germany, Hong Kong, Ireland, The Netherlands, Norway, Scotland, Sweden, Switzerland, U.K. and the U.S.A. — provide the international perspective. The countries represented on the board are incredibly disparate in their linguistic registers and in their cultural and social histories, yet despite the various administrations they represent, are united in an attempt to investigate and discuss issues of common interest.
We acknowledge the ground-breaking work of Paul Black and Dylan William, two of the architects of ‘Assessment for Learning’ (AfL) in the U.K. In their article these authors are, however, uncomfortable with both the narrow applications and wide interpretations of AfL. They worry that a theory of formative assessment may too easily become a panacea or ‘a theory of everything’, so bypassing the essential point which is pedagogy. One aim of their paper is, therefore, to fill a conceptual gap in the literature and to help define the precise location of formative action within a comprehensive theory of pedagogy. Any theory which meets that aim, must, they argue, map the territory that brings into relationship three spheres — the teacher’s agenda, the internal world of each student, and the inter-subjective dialogic process. While feedback is only one aspect of pedagogy, what the teacher does at that critical moment in response to a student’s statement or question, typically with little time for reflective analysis, may speak volumes as to professional judgement and what it reveals about the student’s thinking and motivation. The two steps involved, the diagnostic, in interpreting the student contribution and the prognostic, in choosing the optimum response both involve complex decisions, often taken with only a few seconds available. Understanding of that critical interplay, and the circumstances in which it occurs ‘cannot prosper if the up-stream planning does not provide a context favourable to it’.
Gert Biesta reminds us of the pivotal question ‘What is educationally desirable?’ What is effective may not be ‘good’. The two questions ‘Effective for what? and ‘Effective for whom?’ are questions of quality and equality but too often obscured by a focus on ‘outcomes’, the intrinsic values and purposes of which are too little open to question and critical analysis. Attainment outcomes are the stock in trade of TIMMS, PIRLS and PISA, studies whose impact on policy priorities and school practice have been profound, and profoundly narrowing of the educational agenda.
The reasons for the relative absence of attention to questions about educational purpose are, he contends, not merely ‘external.’ They have also to be explained by transformations within the field of education and a shift in the vocabulary which infuses talk about educational processes and practices. ‘We shouldn’t underestimate the ways in which language structures possible ways of thinking, doing and reasoning to the detriment of other ways of thinking, doing and reasoning’, he writes.
Biesta concludes, ‘If we do not tackle the questions as to what constitutes good education we run the risk that statistics and league tables will make these decisions for us’. This brings us full circle to questions of assessment for learning, critical response to data and the importance of learning conversations. It as important for the everyday practice as for the more explicit evaluations that take place at times of inspection, suggests Biesta, and critical in reporting both on a proactive basis and when schools are called to account for our actions and decisions. The big questions we are left with is accountability for what? Who sets the agenda? Where does the power lie? Should internal accountability framed by what we as educators value drive external accountability? Or should we allow ourselves to be driven by it?
In their article, Ben Levin and Carol Campbell argue that there is, or ought to be, an integral, synergetic relationship between internal classroom assessment and external accountability In the article they draw on large scale reform in Ontario schools. Based on evidence from reform efforts there they argue that building internal accountability linked to external accountability is a key element in successful educational change strategies. Professional accountability, they argue, is based on data, not as a summative final judgment but as an essential part of the toolkit for understanding current performance and formulating plans for reasoned, and reasonable, action. The use of data for accountability purposes is, however, not to treat it as a static numerical snapshot but as a continuing conversation. Data need to be used to stimulate discussion, challenge ideas, rethink directions, and monitor progress. When this is internalized, they write, educators become more assessment literate, less wary of data and more comfortable with their external accountability.
Pedagogy is the focus of the two last papers, which examine the nature of intervention but reach different conclusions about the efficacy of school and classroom interventions. Peter Tymms’ paper examines the effects of early intervention at classroom level, with a particular focus on for children in disadvantaged circumstances. He concludes that a key ingredient in children’s success is the first teachers they encounter whose impact may be powerful but only if sustained year on year by successive ‘boosts’. Much more typical, however, in pupils’ experience is a mixture of boosts and setbacks, the progressive impact of the latter making every boost more difficult to sustain. Progress in achievement relies on a series of positive experiences over a sustained period but this, he argues, is hard to find within a system where boosts and setbacks inhibit the smooth trajectory of improvement. Year on year improvement, beloved of policy makers is, he argues, illusory.
Where Tymms parts company with Mei Kuin Lai, Stuart McNaughton, Helen Timperley and Hsiao Selena’s article is in their conclusion that the academic success of a school is the sum of the successes of its teachers and that the sum is not greater than the parts. Each unit within the school, including teachers, acts by itself, but is part of the whole through loose links. Karl Weick’s loose coupling on which they draw has been well rehearsed in the literature and reinforced by findings from school effectiveness studies. However, as the authors find through case studies of schools in the New Zealand context, there is a whole school effect but one dependent on whether the organization is able to learn how to be adaptive to emerging issues. While a routinised approach supported individuals to be technically competent within their own sphere of operation, the case study school with an adaptive approach was collaborative, exceeding classroom boundaries and helping the school as whole to be a learning community. Rather than applying processes in a routine way teachers were encouraged to theorize their practice, to integrate their knowledge from a variety of strategies and theoretical approaches so as to address a specific issue rather than to seek generic solutions. By extending their own professional reading, by exploration of untapped resources, through conversations and relationships with parents, the adaptive school identified ways in which home learning and school learning could hold the key to improve, and sustain, literacy and self-efficacy.
Both educational practitioners and researchers live in a policy world too impatient for the quick fix and too intolerant of complexity and nuance when it comes to questions about schools’ efficiency and effectiveness. As editors of this journal who have long wrestled with these questions and tried to find sustainable answers, we still thrive on insights that probe the obvious, which test conventional wisdom and shine new light on policy, practice and theory. We welcome contributions which help to break new ground, which take our thinking forward and help to reframe the relationship among assessment, evaluation and accountability in education.