Introduction

Public inquiries can be defined in simple terms as temporary working groups created, mandated and made independent by governments in order to fact-find, hold actors to account or develop policy lessons (Marier 2009, 2017). In the aftermath of policy and political failures, governments routinely rely upon public inquiries to identify lessons which, if implemented, should prevent future failures from reoccurring. Consequently, social science has a rich tradition of examining these institutions, which stretches back to the early twentieth century. Within that tradition, a widespread view exists that the public inquiry is an ineffective means of lesson-learning. Political scientists, for example, consistently claim that inquiries are tools for agenda management which prejudice lesson-learning (Clokie and Robinson 1937; Acland 1980; Prasser 1994). Public administration and law scholars have identified inquiries as theatres in which the politics of blaming undermine attempts to learn (‘t Hart and Boin 2001; Boin et al. 2008; Eburn and Dovers 2015) and sociologists and organisational scholars have both argued that inquiries are either ceremonial mechanisms that are established to restore faith in the status quo (Ashforth 1990; Gephart 2007) or interpretative mechanisms that manufacture, rather than faithfully record, history (Brown 2004; Boudes and Laroche 2009).

Canvassing literature of this nature leads to the conclusion that inquiries are not particularly effective in terms of the identification of appropriate policy lessons. Writing from a risk management perspective, for example, Lauder (2013: 231) tells us that inquiries ‘provide little that is new and what is new is often hidden within the mass of data presented. This might suggest that they do not really understand what is required to fulfil their remit to provide lessons’. However, even if inquiries do identify valid lessons, political scientists still suggest that ‘their political impact varies significantly. Some commission reports set off substantive policy changes; others end up in the dustbin. In fact, when it comes to enacting meaningful reform, the latter outcome (the dustbin) appears to be the norm’ (Parker and Dekker 2008: 255). Such views are also reinforced by an abundance of more public commentaries in which inquiries are regularly vilified as costly wastes of time that illuminate very little (for example, The Telegraph 2004; HC 2005; The Times 2017).

However, an initial examination of the public inquiry literature reveals that conceptual and methodological problems within that research undermine much of our knowledge about the public inquiry. These issues cast doubt on what we think we know about inquiry effectiveness. Therefore, this article reinvestigates the claim that inquiries are ineffectual lesson-learning mechanisms. It does this by first setting out an analytical framework that combines classic and contemporary policy learning typologies and applying that to an international comparison of four inquiries in Australia, Canada, New Zealand and the UK, which was operationalised by thick case study descriptions that utilised 100 interviews with inquiry personnel, senior and junior ministers, policy officials and public sector leaders.

Contrary to the conventional wisdom outlined above, the findings illustrate that each inquiry managed to produce a large volume of valid policy lessons, which subsequently propelled several large-scale and meaningful policy reforms. The data therefore underpin a big picture argument that inquiries may be more effective at learning than we currently appreciate. However, underneath that, more granular data are also presented about the types of learning that these inquiries produced and what this meant for the avoidance of future policy failure.

The article proceeds conventionally. Directly below inquiry scholarship is critically analysed. Thereafter, a theoretical response is outlined in the form of the analytical framework. The research design and methodological features of the analysis are then presented, and the findings follow.

Failing to learn: The state or the academy?

Two significant issues suggest that we ought to be sceptical about views that suggest that the public inquiry is an ineffectual lesson-learner. The first issue relates to a lack of robust data to support that indictment, which is an issue that is primarily caused by a range of methodological weaknesses in the extant inquiry literature. The second issue is more conceptual, relating to the lack of governance and public policy research that has been applied to the study of the inquiry, which has meant that it is often studied without cognizance of its contemporary nature in the modern state.

The lack of data is a significant issue caused by several characteristics of the research effort in this area. First, there seems to be a reluctance in inquiry scholarship to develop primary data through surveying, interviewing or observing those involved within lesson-learning episodes. Exceptions to this rule exist in the form of Rowe and McAllister (2006) who use a small-n of 20 interviewees to examine the internal workings of British royal commissions and Gephart et al. (1990) and Vaughan (1996) who both offer insightful observations off the back of ethnographic fieldwork. However, these small exceptions exist within a larger universe of inquiry scholarship that has tended to eschew primary data completely in favour of ‘light’ case studies that draw their evidence from secondary sources. Second, a great deal of what we know about inquiries comes from the previous century via a form of historical constitutional scholarship, primarily located in public administration and law, which has relied upon biographical essays and anecdote (for example, Benson and Rothschild 1982; Salter 1989; Howe 1999) and forms of analysis that have tended to shun theoretical underpinning or research design (for example, Chapman 1973; Bulmer 1980). This form of scholarship, moreover, reflects a predilection towards understanding the minutia of inquiry format and internal design at the expense of analysing policy outcomes (for example, Stevens 1994; Beer et al. 2011). Third, claims about the ineffectiveness of inquiries can be found most frequently in research that is primarily interested in crisis management and post-crisis politics (‘t Hart and Boin 2001; Borodzicz 2006; Drennan et al. 2015). In these works, the problematic nature of inquiries tends to be insinuated rather than properly evidenced (for example, Drennan et al. 2015: pp. 206–219). Finally, there is a lack of comparative work that focuses exclusively on the public inquiry. Two notable exceptions here are Inwood and Johns’s (2014) comparison of Canadian inquiries and Althaus’s (1994) comparison of an Australian and Canadian inquiry into infrastructure projects. Nevertheless, most contemporary research on inquiries can be found scattered across ‘one-off’ studies that have been conducted by scholars who are more interested in the topic that the inquiry is investigating rather than the lesson-learning process itself (for example, de Brujn 2006; Ralph 2011). What we therefore lack are findings about the inquiry-lesson-learning relationship that have value beyond the idiosyncrasies of single-case specifics. This issue is compounded by the fact that political scientists, organisational theorists, sociologists and legal scholars all examine inquiries with little cognizance of the work of others outside their field (Marier 2017).

These issues mean that any claim that inquiries are ineffectual lesson-learners needs to be treated with a degree of scepticism. However, conceptual issues also undermine our knowledge of inquiries. A significant problem here is that a great deal of inquiry scholarship has not kept pace with developments in the various sub-fields of political science, most notably those concerned with governance and public policy. The single biggest problem in this regard is that inquiry literature continues to treat lesson-learning as something which takes place exclusively in the inquiry room or in the corridors of central government. This basic view of lesson-learning blights our understanding of the inquiry because it is not cognizant of what we now know about policymaking in the modern state. Consequently, two conceptual problems arise. First, as Elliott (2009) has argued, there is a tendency in inquiry scholarship to reduce the process of lesson-learning into an oversimplification that focuses exclusively on how inquiries produce policy knowledge and/or how that knowledge is accepted or rejected by central government. One outcome of this limited view is that many policy-relevant aspects of lesson-learning escape analysis simply because they exist ‘beyond’ these two stages. The second conceptual issue is that a range of influential agents, widely cited as important in the policy learning literature, are neglected from the analysis of inquiries, despite their potential importance. Thus, the role of scientific communities (Haas 1992), advocacy groups (Weible and Sabatier 2007) and proximate agencies in other countries (Rose 1991), for example, have all been ignored in studies of the inquiry even though they have been defined as important factors in policy learning generally. These issues highlight that what we now know about governance and policy in the modern state—namely that it is a crowded and complex affair—is not being properly factored into analyses of the inquiry.

When we stack the conceptual problem on top of the lack of primary data, we have grounds to question any claim about the effectiveness of these mechanisms. This article therefore returns to that fundamental research question, which has been asked but not properly answered: can public inquiries effectively learn lessons in ways which can reduce the likelihood of future failures? The intention in what follows is to address that question with a response that is underpinned by theory and evidence.

Theorising public inquiry outcomes

In this section, classic and contemporary studies of policy learning are used to build an organising framework that has the capacity to evaluate inquiries in relation to policy outcomes. Before we begin this work, two points about the sub-field of policy learning are worth noting. First, it has traditionally lacked cohesion. Although certainly sophisticated, the field contains a dizzying array of definitions, foci and theory, all under the rubric of learning, yet orientated towards rather different concerns. Moreover, where there is consensus, it tends to be found in claims that the field is conceptually ambiguous, in need of greater integration and altogether a bit too complex for its own good (Huber 1991: 89; Levy 1994: 280; Birkland 2004: 344; Schofield 2004: 291; Dunlop and Radaelli 2013: 600). Hence, we are told, for example, that getting to grips with policy learning is akin to sweeping ‘a conceptual minefield’ (Levy 1994: 279); that the literature’s ‘guiding concept is characterised by confusion and scholars find the phenomenon hard to define, isolate, measure and apply’ (Deverell 2009: 180); and that the differences between the major works in the field mean that ‘the entire phenomenon of experience-induced policy change remains difficult to operationalise’ (Bennett and Howlett 1992: 276). There is therefore a need for parsimonious theory, which can drive an analysis of the inquiry in a straightforward manner. The second point about policy learning theory is that things are getting better in this regard. While the classic macro studies of policy learning, such as Heclo (1974) and Hall (1993), rarely make sense in relation to each other, more recent ‘2.0’ studies have drawn together the disparate threads of the field into a more cohesive fabric (for example, Freeman 2006; Grin and Loeber 2007; Dunlop 2017; Dunlop and Radaelli 2017). These works can help us cut a path through ‘the minefield’ of learning literature so that we can be clear about what we mean when we use the term policy learning in relation to inquiries.

The framework that is unfolded below has several features. First, it only utilises those policy learning types from organisational and policy scholarship that appear, theoretically, to be most germane to the evaluation of an inquiry’s lesson-learning outcomes. Second, it combines multiple types of learning together within three broad categories, which are defined in terms of formal-institutional, cognitive-interpretative and value-orientated learning. These three categories allow us to simplify the morass of different types of policy learning in a parsimonious way, which nevertheless capture the range of learning outcomes that inquiries are likely to produce. Third, the purpose of the categories is to provide a framework for analysis that has the capacity to determine and evaluate the nature of different kinds of inquiry outcome. It is intended to provide a simple and transferable tool for comparatively analysing what learning outcomes inquiries produce and the influence of those products in terms of policy reform. Thus, the framework takes pre-existing policy learning types, stylises them theoretically to the inquiry context and then applies them in this new empirical area in order to produce novel findings.

Cognitive-interpretative learning outcomes

All learning begins with the individual (Rose 1991: 7; Sabatier and Jenkins-Smith 1993: p. 123; Dunlop and Radaelli 2013: 600) and to ignore that means severing the starting point of any learning process from the analysis (Dunlop and Radaelli 2017, pp. 307–309). Theoretical discussions of policy learning ought therefore to at least acknowledge that learning begins with a change in an actor’s beliefs, insight or understanding (Levy 1994: 287), which can result as part of a conscious choice (Friedlander 1983: 194) or an unintentional process (Huber 1991: 89). However, the most relevant policy learning definitions, at least when it comes to inquiries, are those that emphasise the collective rather than the individual. We can make this claim simply because the production of policy via an inquiry is, by default, an inter-organisational activity (involving, at the very least, an independent inquiry and the relevant government machinery). As a general departure point therefore, we can initially use Cook and Yanow’s (1996: 438, emphasis added) definition of organisational learning, which emphasises that when a ‘group acquires the knowledge associated with its ability to carry out its collective activities, that constitutes organisational learning’.

When thinking about collective learning, it is helpful to distinguish between learning that produces formal-institutional gains and learning which produces shared cognitive gains. The latter is what is meant here by cognitive-interpretative learning outcomes. At the most basic level, this can be defined in terms of increases in the degree of shared understanding experienced by a range of different actors. In any inter-organisational context, collective action requires a degree of collective awareness and when actors develop a more sophisticated appreciation of their policy domain, their place within it and how they relate to other agents, cognitive organisational learning can be said to be taking place. This appreciation of the other, developed through a learning process, resonates with classic policy learning scholarship, such as Etheredge and Short’s (1983: 48) seminal view that learning produces ‘collective institutional coherence’ across government agencies and also in more contemporary learning studies that emphasise the relevance of developing ‘shared interpretations’ as a learning outcome (Huber 1991). Both views can be connected to contemporary governance empirically, because we are continually told about the need for coordination in a differentiated governance era (Rhodes 2007) and also theoretically, because of the emphasis that scholars place on understanding how shared interpretations affect policy outcomes (Bevir and Rhodes 2010).

It is therefore theoretically possible for policy learning to occur when groups of policy actors enjoy growths in their ‘collective intelligence’ which better connect them to other actors in their policy space. In the case study comparison that follows, if this coherence emerged because of a public inquiry’s lesson-learning, then the label ‘cognitive organisational learning’ was applied and the analysis sought to determine what effects it had on the relevant policy area.

Formal-institutional learning outcomes

Collective gains in shared understanding can be contrasted against a more mainstream view, which suggests that learning outcomes ought to be measured in terms of changes to the formal-institutional features of a policy space. Barbara Levitt and James March’s (1988: p. 320) seminal work on organisational learning, for example, suggested that organisations learn when they encode ‘inferences from history into routines that guide behaviour … in a way that makes the lessons, but not the history, accessible to organizations and organizational members who have not experienced that history’. This view suggests that cognitive forms of organisational learning need to be complemented with another type—institutionalised organisational learning—which is when identified inquiry lessons get hardwired into the organisational routines and memory of government agencies in ways which influence collective behaviour.

A more specific type of institutional learning is defined by Peter May as instrumental policy learning which ‘entails new understandings about the viability of policy interventions or implementation designs’ (May 1992: 335). This type captures the ideal of policy learning in which evaluation leads to redesign and redesign leads to new or improved policy instruments. The primary learners here are policy officials, and the underpinning logic is rational analytic. In order to evidence this type of policy learning, May tells us that cognitive changes would consist of an ‘increased understanding of policy instruments or implementation’ and behavioural change would be seen in ‘policy redesign entailing change in instruments’ (May 1992: p. 336). In simple terms, therefore, we would expect to see instrumental inquiry learning when an inquiry propels a change of thinking in government, individually or collectively, which can be connected to the production of new or revised policy tools.

In a similar manner, the concept of single-loop learning draws our attention to a type of learning that focuses upon observable institutional change (Argyris and Schön 1978; Argyris 1982). A single-loop learning approach suggests that we can learn and correct errors without questioning the values that sit beneath those errors. Thus, the object of learning is something ostensible such as the policy instrument, policy design or the standard operating procedure, for example. However, the application of a single-loop label also means that the values that sit beneath efforts to learn about these objects will not be questioned. In this context, reflection on the learning process itself will be absent. This limitation means that ‘whatever learning people develop will tend to be within the confines of what is acceptable. … Few people will confront the validity of the goal or the values implicit in the situation’ (Argyris 1982: 88). Consequently, single-loop policy learning outcomes are often viewed negatively. Nevertheless, in the context of an inquiry, single-loop learning will be evidenced in an abundance of narrow, technical (but very observable) lessons that change the formal-institutional componentry of a policy space.

Value-orientated learning outcomes

In contrast to single-loop learning, double-loop learning outcomes will be reflected in more radical changes in the values, assumptions and cultural norms that characterise policy spaces. This is because double-loop learning is said to be more reflective, which means that it encourages a willingness to scrutinise and challenge the meanings given to information in a constructive and open manner (Argyris 1982: 103). This willingness, moreover, means that ‘the basic assumptions behind ideas or policies are confronted’ (Argyris 1982, pp. 103–104). Consequentially, double-loop learning outcomes in the context of an inquiry will be reflected in lessons orientated towards values, assumptions and cultural norms. This focus on values can also be associated with another influential definition that is relevant to the inquiry—social learning.

Social learning ‘entails a new or reaffirmed social construction of a policy by the policy elites of a given policy domain’ (May 1992: 337). The provenance of this term stretches back to the early social psychology of Bandura (1977) who was one of the first to explore the means through which individuals draw on social context when learning. However, political scientists have taken the principle that ‘context matters’ and used it to explore the state–society relationship in more macro terms. In this type, the fundamentals of a policy—the nature of the problem, causal beliefs, and the goals and target groups, for example—are reconceived. This category is one reflection of a larger lineage of works, which have developed the concept of social learning more broadly (Heclo 1974; Hall 1993), and we need to be careful here because these authors often talk at cross-purposes (see Bennett and Howlett 1992), but, generally, they focus upon the ways in which value-orientated changes in the social environment propel policy learning inside government by changing notions of appropriateness. For Heclo (1974: 305), this is something of an automatic and gradual process: the daily grind of incrementalism is seen to be a result of the state ‘puzzling’ its way slowly through policy reforms as a reaction to social stimuli. For others, such as Peter Hall, it is more purposive because a range of policy changes, from ‘first-order’ instrumental tinkering to ‘third-order’ paradigm overhaul, can result from a deliberate state response to external pressure (Hall 1993). May’s description of social policy learning reflects Hall’s understanding of third-order ideational change more than Heclo’s. This can be seen in the indicators of social policy learning. At the cognitive level, we are told to search for ‘change in dominant causal beliefs within the relevant policy domain’ and at the organisational level ‘policy redefinition entailing change in policy goals or scope’ (May 1992: 336). In an inquiry context, therefore, social learning will be seen if an inquiry propels a change of thinking amongst elites about the primary values at work in a policy area.

These three broad categories of learning outcomes, and the more specific types within them, provided an organising framework that was subsequently applied to the case study comparison. The design of that comparison is outlined below.

Research design and methods

In order to address the research question and respond to the critiques of the public inquiry research set out above, a number of specific design and methodological decisions were taken. The first major decision was to pursue data via qualitative means, principally through interviews, because of the cognitive-interpretative and value-orientated dimensions of the theoretical framework. These aspects required attention to be given to capturing deeper and more subtle meanings of the kind that qualitative interviews are most suitable for delivering (Chase 2013: 60; Landman and Carvalho 2017: 26).

The second major design decision was to use interviews in conjunction within a larger process of ‘thick description’ (Ryles 1971; Geertz 1973; Denzin 2011). Although first promoted by ethnographers as a means of inviting culture and self-reflection into data gathering (see Geertz 1973), the thick descriptive process has mainstreamed into qualitative research more generally as a way in which a researcher can combine rich empirical description, the lived experiences of participants and their own analytical interpretations into a single process (Ponterotto 2006). Thus, ‘to thickly describe social action is actually to begin to interpret it by recording the circumstances, meanings, intentions, strategies, motivations and so on that characterise a particular episode. It is this interpretive characteristic of description rather than detail per se that makes it thick’ (Schwandt 2001: 255). The value of thick description relates to the way in which it can deliver an in-depth wealth of data about a topic (Ellingson 2013: 433) which connects lived experiences to public policy decisions (Denzin 2011: 3). We need to be quite clear, however, that the interpretive dimension within this process precludes the use of terms such as reliability and replaces them with others, such as trustworthiness, which better recognise the nature of qualitative research (Heaton 2004: 100). Nevertheless, there are many different degrees of thick description, which can accommodate different methodological positions, and the process can sit alongside the deductive use of pre-existing theory and more structured analytical frameworks such as those used here (see Denzin 2011, pp. 99–118).

The final major design decision was to use an international comparison of four cases. A comparative analysis was chosen for several reasons. First, because of the theoretical framework’s orientation towards developing an understanding of the different types of policy learning that an inquiry can produce and its orientation towards assessing the view that inquiries are weak policy learners. Comparative institutional analysis is regularly promoted on both counts as a means of developing types (Peters 1998: 10; Lees 2006: 1099; Landman and Carvalho 2017: 4) and as a means of ‘checking’ the validity of dominant interpretations (Hopkin 2010: 290). Second, a comparative approach was chosen because of the wish to avoid the kind of cultural exceptionalism that often accompanies single-case studies (Lees 2006: 1098). While the use of qualitative methodology negates the kind of abstraction that is achieved by quantitative means, the use of multiple cases does reduce the degree to which findings are uniquely context dependent. Finally, a small-n case comparison such as the one used here also sits well with the thick descriptive process because it can deliver a large amount of granular data that are essential to uncovering deeper degrees of meaning (Ragin and Rubinson 2009; Landman and Carvalho 2017) and because it also encourages a tacking ‘back and forth between theory and evidence’ (Ryan 2018: 281).

Four post-crisis inquiries were explored: (1) the Canadian SARS outbreak of 2003 and the SARS Commission; (2) the UK summer floods of 2007 and the Pitt Review; (3) the 2009 Australian ‘Black Saturday’ bushfires and the Victorian Bushfires Royal Commission (VBRC); and (4) the 2010 Christchurch earthquake and the Canterbury Earthquakes Royal Commission (CERC). The cases were chosen for two reasons: first, they represent very different public inquiries. The Pitt Review was run by public managers using the tools of policy research. The SARS Commission chose to utilise an Accident Investigation Board model. The Royal Commission into the Victorian Bushfires employed a mixed methodology that merged community consultations and novel types of hearing. And the Royal Commission into the Canterbury Earthquakes took a predominantly legal–judicial route, which was complemented by formats designed to encourage the use of technical, engineering logic. In this sense, the research design is predominantly ‘most-different’ in nature because it is calibrated to look for similarities, in terms of types of learning and learning outcomes, across four very different inquiries (Seawright and Gerring 2008: 298). This aspect of the design enhances the article’s findings in terms of different types of learning because they emerge from shared themes generated across difference (Landman and Carvalho 2017, pp. 74–75; Ryan 2018: 279). A second reason for the case selection was that after these inquiries reported and their reform processes concluded, similar crises arrived to test those reforms. What we therefore have in each case is a chance to see not only what types of policy learning materialised but also what effect learned-lessons had after they were implemented. This dimension of the cases allows us to address the more fundamental question of whether inquiry learning can help avoid failure.

Exactly 100 interviews were conducted. The distribution was broadly even in number: UK (28); Australia (28), Canada (22); New Zealand (22). The interviews were semi-structured. The interviewee sample reflects the polycentric nature of the learning episodes themselves. In each case, three broad groups of actors were interviewed. First were those who were involved in the design and staffing of the inquiries (35 in total). For example, commissioning ministers, lawyers who drafted terms of reference, the chairs themselves, assisting barristers and lawyers, and seconded public officials. The purpose of interviewing this group was to develop an understanding of how those ‘on the inside’ viewed the process and outcomes of policy learning. Second, were those directly responsible for the implementation of inquiry lessons (55 in total), primarily central government ministers and officials, local government officials, members of the emergency services, front-line public sector workers and members of non-government and not-for-profit agencies. With this group, the requirement was to find out whether reforms had occurred which institutionalised the inquiry lessons, what those reforms looked like and, crucially, whether there they had reduced vulnerability to future threats. Finally, a range of non-government experts (15) were used to corroborate the claims of inquiry personnel and officials. This final group represented a range of experts who were not directly involved in the lesson-learning process per se but who are nevertheless still heavily invested in each relevant policy area.

The interview transcripts were coded via a simple process in which apposite data were first ‘sorted’ into the various categories of the framework. For a learning type from the framework to be confirmed, interviewees had to initially connect a change in cognition, individually or collectively, with the inquiry. This meant that something had been learned initially. That lesson then had to be connected to a change in policy related to the categories set out above—shared understanding, institution/instrument or value-orientation. Thereafter, documentary evidence was searched for again to substantiate the claims about outcome. Following these two steps, a grounded theory approach was used within each learning type category to build a more nuanced picture. This involved a process that created a subset of categories inductively (see Strauss and Corbin 1990; Charmaz 2000). For example, as the cognitive organisational learning data grew across cases, grounded coding identified sub-themes in relation to the identification of departmental silos, specific coordination mechanisms and individual coordination champions. Through this process, cross-case themes emerged and the largest of these are reported below.

Analysing public inquiry outcomes

Three cross-case themes stood out clearly in the data. First, these inquiries produced significant amounts of instrumental learning, which can be associated with improved policy responses during future crises. It is the strength of these cross-country data that allow us to challenge the conventional wisdom that public inquiries are ineffectual drivers of policy reform. Second, significant evidence of cognitive organisational learning was also found. These findings resonate strongly with Etheredge and Short’s (1983: 48) classic view of policy learning as a means of generating ‘collective institutional coherence’. This form of learning can be associated with specific policy changes through which crisis management agencies better coordinated joint action in relation to pre-crisis planning. The final pattern relates to the absence of data in support of the value-orientated learning category. Thus, each inquiry showed a preference towards single-loop rather than social or double-loop learning.

Instrumental learning outcomes

It is worth recalling here that several connections had to be present before instrumental learning could be properly established. Behavioural changes, in the form of new or revised policy instruments, had to be linked to prior cognitive changes about the need for new instruments, and both needed to be specifically associated with the work of an inquiry. These connections were made repeatedly across all four cases. The challenge was then to determine what, if any, effects those outcomes have had on future crises of a similar nature. Brevity does not permit a full account of the abundance of data that show how instrumental learning produced positive outcomes in this regard (see Stark 2018). Nevertheless, several examples highlight this impact perfectly.

In the UK, for example, every interviewee connected the Pitt Review to flood management tools that improved the government’s capacity to map flood risks, forecast floods and warn citizens in a timely manner. In this regard, the Flood Forecasting Centre (FCC) was ubiquitously cited as an example of a successful lesson identified by an inquiry, learned and implemented by government, and then tested successfully in a future crisis. The FCC brought together a small team of hydrologists from the UK Environment Agency with meteorologists from the UK Met Office. Those involved in the creation of the FCC were in no doubt that a learned-lesson from the Pitt Review was crucial to its creation:

The thing about inquiries looking back—especially Sir. Michael Pitt—is that it does connect organisations better. So independent organisations, if they do lessons learned, will always do reviews but they tend to do them independently. They bring their organisational learnings into themselves. An independent inquiry pushes connections, so if the Met. Office had done their review in 2007, the Environment Agency had done their review in 2007, would we have gone as bold as the Flood Forecasting Centre? These things are usually impossible but the inquiry helped enable that. (Environment Agency Official A, interview)

Between December 2013 and February 2014, England and Wales experienced a second significant flooding emergency caused by highly unusual clustering of twelve intense storms. A combination of exceptional rainfall, gale force winds and a 19-year high tide, created river, groundwater and tidal flooding with high energy surges across 3000 km of coastline. These became known as the 13–14 Winter Floods in which 11,000 properties were flooded, but no casualties recorded. While the impact of these events was serious, interviewees consistently made the case that the capacity of the system to better forecast and warn saved lives and property. Inside the Environment Agency, for example, the FCC was said to be ‘the obvious real plus that came out of Pitt. It was a clear recommendation that was implemented immediately, and it worked’. (Lord Smith, Former Chairman of the Environment Agency, interview). One senior official who was at the centre of the 13–14 response conveyed just how important the FCC’s detection capacity was:

In our ensemble forecast, you could see every model showing low pressure in the North Sea at the same time as the high tides and then there was the prediction of the storm. Now, what was fascinating, as a bit of a geek, I grabbed my iPad and I looked for a satellite image to see where this whirling storm was now before it hit us and the thing that I’ll always remember is that there was no storm at that point because it had yet to form! They were giving us a warning, but this thing only existed in the supercomputers. It had yet to form in the Atlantic and then move over and hit us. We effectively managed to mobilise around a thing that didn’t exist. (Environment Agency Official A, interview)

While it would be tempting to write off these views as self-serving narratives presented by Whitehall elites, there was also widespread support for this view outside of central government. In the emergency services, for example, it was recognised that the FCC had ‘worked brilliantly, helping to translate what the scientists were saying into the right language for the emergency services control rooms’ (UK Fire Chief A, interview). Other interests praised the Environment Agency for having ‘done all sorts of wonderful things in terms of forecasting and modelling and prediction’ (Former National Farmers Union Official A, interview), while in the non-governmental National Flood Forum, the 13–14 crisis response was described as ‘phenomenal, thousands and thousands of people weren’t flooded due to improvements. Not many more were flooded than in a normal year. In terms of an exercise in flood management, it was incredibly successful in unprecedented circumstances’ (NFF Chief Executive, interview).

The Victorian Bushfires Royal Commission also produced a great deal of instrumental learning. This time, however, an enhanced system for communicating bushfire warnings to at-risk communities was said to be the major improvement to emerge from the VBRC. Indeed, this was ubiquitously cited by all Australian interviewees as the most positive change that materialised from this inquiry. Within Emergency Management Victoria (EMV), the organisation now responsible for bushfire response policy, these changes were clearly associated with the VBRC:

If you want the community to react you’ve got to be able to provide them information that’s timely, relevant and tailored. If you think about primacy of life and putting information to communities, those two principles in our new control priorities were the game changers. Then it was about putting community centre because that was a criticism in the Royal Commission, a huge criticism in the Royal Commission, that we weren’t focused on getting information out and did not relate to the community. (Craig Lapsley, Victoria Emergency Management Commissioner, interview)

Enhanced bushfire warnings were also directly connected to tangible improvements in emergency response. One policy official in the Department of Environment, Land, Water and Planning (DELWP), for example, reflected on the complexity associated with connecting inquiry recommendations to policy outcomes and suggested in the end that:

You can draw a line of sight between a lot of the individual recommendations and suggest that they are likely to have achieved better public safety outcomes. I don’t think with a lot of them there’s a discipline yet to show that; to actually prove that there are better public safety outcomes, but I think you can in terms of better warnings and information (DELWP Official A, interview, original emphasis)

These instruments have also been tested during subsequent emergencies. The 2012–13 and 2013–14 fire seasons in Victoria, for example, involved a significant number of serious bushfires. The Bushfire Royal Commission Implementation Monitor appraised the state’s response to these fires, highlighting that ‘public information, warnings and advice are now far more comprehensive and concise than they were on Black Saturday’ (BRCIM 2014, p. 14). Interviewees, however, focused on another significant bushfire that emerged in the south of the state more recently. In the Wye River and Separation Creek region of Victoria, several fires threatened life and property in 2015. There was a unanimous view within the data that the response to these fires highlighted warnings and evacuation working well:

We lost 116 houses at Wye River and Separation Creek, which is a major beachside resort area. Really noteworthy through that whole exercise was that whole settlement was successfully evacuated. So, given the emphasis on the primacy of life, on which I think the Commission was absolutely right, no one died. (Emergency Management Victoria Official A, interview)

Once again, these views were validated outside of central government. In the emergency services, it was recognised that the Wye river response had shown the effectiveness of ‘cascading, regimented and clear warnings’ (Victoria Police Commissioner, interview). In the independent Bushfire and Natural Hazards Research Centre, the changes to warnings were regarded as ‘important and substantial’ (BNHRC Official, interview) and in Victorian local government, the 2015 emergency response was praised because:

No lives were lost. I think the messaging from the state control centre through to the local brigades through to the general community worked really, really well. You could see that; you could see how well coordinated it was. Yes, there was a lot of panic and those sorts of things, but no lives were lost. (Victoria Municipal Official A, interview)

In New Zealand, a rapid, post-earthquake building inspection system was defined as the most important instrument to emerge from the CERC lesson-learning. The Commission proposed no less than 50 detailed recommendations around post-earthquake building inspections and concluded that a new inspection system ought to be institutionalised via legislation. What has emerged from government in response are a standardised series of policy instruments including a new ‘traffic light’ placard system, standardised usability assessment tools and a series of field guides for evaluators. These were accompanied by training tools designed to help local authorities implement the new system. The former Ministry of Business Innovation and Employment (MBIE) official who took carriage of the implementation process confirmed the connection between this example of instrumental learning and the CERC:

When I left MBIE, I had completed a manual, completed all the resource material and completed all the training material and produced two quite chunky field guides for staff and I was delighted as a former colleague said to me “we’ve all been trained and that material, it was great – it hit the spot”. To discover that people were being trained and that it was working was wonderful. (Former MBIE Official A, interview)

This specific set of instruments was put to the test in July 2013 and November 2016 during which the capital city of Wellington was shaken by earthquakes that prompted the evacuation and inspection of its central business district. The November 2016 earthquake, in particular, was significant. It measured 7.8 magnitude, killed two and damaged hundreds of buildings. Once again, the Wellington CBD was evacuated and a series of buildings, some with significant damage, had to be inspected before being reopened. Initial media reports praised the post-building inspections (see, for example, The Guardian, 8 February) and officials in MBIE (which is based in Wellington) and Wellington City Council both emphasised once again that the inspection system had worked well:

No question, dramatic improvement. How have the assessments gone for this event? To give you an idea, by and large all the assessments for Wellington are already finished and this is the real positive; the recording system and the forms that were developed post-Canterbury worked really, really well and the assessments have been done really, really quickly. (Wellington City Council Official, interview)

In the SARS case, the revision of epidemiological surveillance tools was consistently presented as an example of an important lesson-learning gain. This was said to have resulted in a situation where disease outbreaks occurring elsewhere could be better monitored, new infectious disease could be identified clearly when they did arrive, and that they could be subsequently tracked much more effectively should they escalate. As far as the Province’s Cabinet Secretary was concerned, this was a significant gain to emerge from the SARS Commission:

Surveillance, data and infection control. The true story of SARS that is very graphic is of public health officials trying to track cases using a system of coloured ‘stickys’ on walls – paper and pencil stuff – and there was not a broad enough provincial capacity or architecture for holding data of that sort and for rapidly tracking where cases were ending up and tracking them back to source. That was being done by paper and pencil so in terms of lessons learned and things that are, for the most part, fixed; the capacity for case tracking is top of mind for me because that capacity was pretty non-existent. (Tony Dean, Former Cabinet Secretary, interview)

In 2009, an H1N1 pandemic killed over 17,000 across the globe. Ontario’s response to H1N1 was said to have reflected an effectiveness that could be traced to the SARS Commission. In the public domain, for example, new investments in laboratory capacity and more effective surveillance tools were repeatedly cited as evidence of SARS learning. Ministers, public health officers and a wide range of independent observers can all be found on the public record, praising the improvements in surveillance in relation to H1N1 (see, for example, Toronto Star 2009a, b). Similar views were expressed again in relation to the Province’s 2014 response to the potential threat of Ebola (Calgary 2014), and of course, these views were echoed in interviews. The Chief Medical Officer for Public Health in Ontario, for example, underlined how much had changed since SARS:

we have such sensitivity now that we can pick things up early but we also have specificity now too. So, the pandemic. We did intense surveillance and rapid laboratory testing of the initial clusters and we were the first ones worldwide to identify that people who were born before 1957 had a significant immunity to this novel pandemic. That meant we could step back and breathe as we knew that a large proportion of the elderly would not be dying. That breath gives you focus, you work on your coordination, your communication and we were successful (David Williams, Chief Medical Officer for Public Health, interview)

Once again, the views of government were validated by independent interviewees. Dr. David Walker, for example, who chaired the SARS Expert Panel and also investigated the performance of the subsequent policy reforms, cited increased surveillance capacity as one of the key benefits to emerge from the lesson-learning process (David Walker, interview). At the local level, public health officials praised Public Health Ontario’s (PHO) willingness to provide them with surveillance data and information that catered to local needs (Regional Public Health Official A interview) and in the Registered Nurses Association, the PHO’s ability to provide more accurate incidence measurement was seen as one element of a suite of reforms that had enhanced clarity around outbreaks.

Cognitive organisational learning

When the shared cognitive capacities of a group of inter-organisational policy actors develop, cognitive organisational learning can be said to have occurred. For cognitive organisational learning to be established in the cases, however, interviewees had to connect inquiry lessons to a greater level of shared understanding between policy actors. Questions were then asked about the outcomes that cognitive cohesion produced, particularly in relation to the more tangible elements of policy and their performance in future crises. In three of the cases, significant evidence of cognitive organisational learning was found. This was defined through the claims of actors who emphasized that they had become more aware of their policy community, their place within it and ways in which they had to work alongside others to achieve collective crisis management outcomes. Moreover, this type of learning was said to have changed organisational routines in ways which enhanced collective working. Thus, cognitive learning facilitated formal-institutional learning as greater coherence was hardwired formally into different policy spaces.

What is remarkable is that each inquiry sought to solve the issue of fragmentation via the creation of new offices or the strengthening of pre-existing officeholders who could promote greater cohesion. Thus, new leaders were created, given clear crisis management responsibilities and they subsequently became coordination ‘champions’ in their respective policy spaces. In Victoria, for example, the VBRC identified coordination problems with central command and control, operational incident command, the interoperability of emergency services and the relationship between the state and federal tiers of government (VBRC Final Report Summary 2010, 8–10). Consequently, it proposed that an independent Fire Services Commissioner be created, made accountable to the Victorian Parliament and mandated with the responsibility of ensuring interoperability in fire response. The establishment of this office, which later broadened out to become the Emergency Management Commissioner, was validated as a lesson-learned which had brought together the emergency management community of Victoria. According to the Bushfire Commission’s Independent Monitor, for example:

It was the creation of that role which for the first time really drew together a central point of authority and control. You may recall that one of the key issues that came out of the Royal Commission was the question of who was in charge at critical times during the fire, and because no one was clearly designated as the person in charge that allowed some of the shortcomings that were reported on to occur. So to me, all of the other things - there are many things that contributed; infrastructure, technology - but the real thing, I think, that makes a difference is having that central effective coordination so everyone knows who’s in charge at any given time and where they should go for high-level decision-making. (Neil Comrie, Bushfire Implementation Monitor, interview)

The metamorphosis of the Fire Commissioner into the Emergency Management Commissioner was accompanied by the creation of Emergency Management Victoria. This is now the central coordinating mechanism for crisis management policy at the state level. At the centre of its efforts is the concept of consequence management, which is a term that appears repeatedly in the Australian data. Consequence management involves tracing the ripple effect of a crisis beyond the immediate priorities of emergency management and crisis resolution. It means anticipating how an event will channel through systemic interdependencies and once that tracking has been done, bringing the relevant agencies to the table as part of the planning process. This concept has allowed the central emergency services to take huge strides in terms of cognitive organisational learning. Indeed, as one police commissioner confirmed, this form of policy learning has changed the emergency management policy space in Victoria:

from a largely unsophisticated approach with silos to a joined-up one with everyone having responsibility. From an operational perspective, what that meant in reality was you had not just a focus on putting the fire out but real consequence management. That meant at the table you now, as part of the emergency management team, you now have transport, education, water, electricity, tourism and from a messaging perspective, what that means to all of that. … So you now have a huge shared responsibility. (Victoria Police Commissioner, interview)

In a similar vein, the SARS Commission also sought to improve coherence and shared understanding in a system which was said to be characterised by silos. The Ontario Province’s Cabinet Secretary at that time recalled:

We had silos baked into the architectural design of the public health system and within that we had live disputes between individuals, tussles for power, in the context of the crisis itself. … That was a stark issue, watching people argue across organisational silos and argue about turf and that was true between individuals and it was true with the province vis-à-vis federal government and it was true in terms of determining what the emergency management people were responsible for versus the Chief Medical Officer of Health and all of this was happening in real time. I have a considerable degree of confidence that you wouldn’t see that repeated again as a result of this inquiry and those reports (Tony Dean, Former Cabinet Secretary, interview)

Like the VBRC, the SARS Commission attempted to cut through the morass of competing interests through a recommendation that a single office—the Chief Medical Officer of Health (CMOH)—takes responsibility for all aspects of infectious disease planning and emergency response. Policy officials once again made the connection between the SARS Commission and the management of future threats:

I can remember the policy exercise of taking each one of Justice Campbell’s recommendations, where relevant to my area of government, doing the analysis, trying to determine the actions that we needed to follow and there are things which Ontario should be very proud of. We immediately amended the public health legislation – the Protection and Promotion Act – newly prescribing the powers and independence of the Chief Medical Officer of Health … those were things which can be directly traced back to Justice Campbell’s reports. The creation of new powers for the Chief Medical Officer to issue directives led to the coordination of communication and these were subsequently used in future events. (Public Health Ontario Official B, interview)

In the UK, there was another complete consensus amongst interviewees that the coherency of the flood management system had improved as a consequence of Pitt-driven reforms. The principle way that this has been achieved is by designating the Environment Agency as the strategic lead for all forms of flooding in the UK and the County Local Authority as the lead agency for operational flood response. To a large extent, this has ameliorated the problem of different floods being owned by different agencies at the local level. Thus, according to one policy stakeholder:

The absolute key to what Sir Michael Pitt said was that people have to work together and, pre-Pitt, what you had was a very disparate bunch of authorities that tended to work in isolation with each other. If it was the wrong colour of water, then it wasn’t your problem. Fundamentally, Pitt has banged everyone’s heads together and effectively said, “Come on everyone, you must all start to think and act collectively about water level management.” It is very difficult to dissociate land drainage with flood risk management, surface water with ground water and river flooding with tidal flooding, and so on. Pitt embraced the essence of the direction of travel needed and where we are now heading today. (UK Association of Drainage Authorities Official, interview)

Growths in inter-agency understanding at the local level were also underlined, crucially both in terms of local planning and local emergency flood response. In terms of the former, one official described his local authority as a unique place geographically:

where there’s potential for groundwater flooding, there’s potential for coastal tidal flooding, rain water, main rivers, drainage systems and the like, but each of the agencies were responsible for each of those sets of defences, if you like, they were not joined up. They only come together now through the lead local authority as it is now. And they were competing in some instances. There were actually schemes being developed by one agency which would significantly increase problems - operating running costs and the like - for another agency. So you now have all the bodies together and it’s having that consistent communication with groups and bodies and having that practice of working together. (UK Local Government Official C, interview)

What this evidence shows is that these inquiries encouraged a disposition amongst policy actors to communicate and integrate more effectively with each other, both in terms of pre-crisis planning and crisis response. The inquiries therefore helped to lay the foundations of a permissive context within which cognitive organisational learning can occur. Moreover, when their recommendations are followed, new or invigorated coordination ‘champions’ are inserted into these policy contexts. These actors can capitalise on the momentum for change and produce the ‘hard’ components—the concepts, the tools and the architecture of coordination—which hardwire policy sectors together.

Value-orientated learning

The one remaining cross-case theme that emerged in the data relates to the lack of double-loop and social learning outcomes. These outcomes were not completely absent from the cases. In Canada, for example, the SARS Commission’s advocacy for the precautionary principle in epidemiological emergencies has now permeated into the public health system and in Australia the VBRC’s conclusion that the primacy of life was not being prioritised in bushfire response policy has propelled a reordering of the hierarchy of goals in that area in a way which reflects Hall’s (1993) social learning theory. However, these two examples stand alone in contrast to the abundance of single-loop reforms that were proposed by these inquiries and then implemented by the respective agencies.

The logical conclusion from this is that inquiries are more likely to produce amendments to pre-existing policy systems than more radical changes to the values behind those systems. This conclusion fits with the preponderance of instrumental learning noted above and also reflects the normative preferences of the majority of interviewed inquiry personnel who emphasised that pragmatic lessons were harder to ‘shelve’ because of their non-threatening nature. Central to these views was a belief that recommendations had to be ‘implementable’ in the eyes of those would assimilate them. This was a belief that was best summarised by one official within the VBRC in Australia:

we were asked to understand what had happened and write recommendations for the future so knowledge transfer was at the core of that and we knew we had to be persuasive so that meant, in our view, the Report being accessible, being consumable, for it to be reflective of the objective analysis but also the lived experience. Did we care about implementation? Yes, we actually had Commissioners with real experience of the public sector who understood how you actually get things up in the service, so yes we kept an eye on implementation but we certainly never thought that it was our job to get them implemented but it was our job to make them implementable. (VBRC Official A)

This focus existed within all the inquiries, and it translated itself into different practices within them. Some inquiries used their interim reports as a means of consulting and communicating with implementing agencies so that they might develop legitimacy for their recommendations. Others costed the expenditure required to implement their lessons and provided implementation blueprints to encourage delivery. Several put in place independent oversight mechanisms to assist with the implementation process or monitored progress themselves. And all the inquiry reports were drafted in a language that recognised the reality of public management. In these ways, inquiry staff sought to ensure that their lessons got transferred and assimilated. This was epitomised by the Pitt Review, which was one of the inquiries that provided an implementation guide and a costing of recommendations. Its Secretary explained that:

if you don’t do that, you let people off the hook. You have to demonstrate that they can do it, and they should do it. … So, how do we make it impossible for them to say no? And that’s why we put so much effort into saying, “here’s the evidence on that. Here’s the scientific evidence. You tell me you can’t do it, here’s a plan for doing it. It’s going to cost you much less, let me tell you how little it’s going to cost.” (Roger Hargreaves, Pitt Review Secretary, interview)

However, this concern with implementation also meant that value-orientated lessons, which had the potential to be more radical or politically challenging, were not pursued. What was notable about this preference for the pragmatic was that it also generated approval amongst those who had to implement each inquiry’s reforms. For example, orientated as they were towards improving rather than overhauling the status quo, the practical nature of the Pitt recommendations was appreciated by implementers. In the words of one local official, for example, ‘Pitt is quite ‘doable’, it’s quite practical … that’s not such a bad thing from this side. If things had been too out of the box, I’m not sure how you would sustain the momentum’ (UK Local Government Official C, interview). And in terms of reducing political contestation the point made below by one former member of the National Farmers Union, a notoriously vocal interest group in the UK, is also important. Here, the case is made that if the Pitt Review had pursued more radical thinking, it would have meant conflict:

I don’t know whether there were was such an appetite for radical thinking … and certainly with those NFU farmers—if you said, “yeah, this is climate change,” you’d have a whole debate on your hands before you actually communicate anything to anybody. And, actually, all you want to do is communicate about behaviour change, to manage risks … there was an element of that within the reports because actually, you would have ended up creating a whole other chapter and I think if they strayed into those areas, all of the headlines would’ve been about “this flood was a result of climate change” or whatever it might be. Actually, you would have missed some of the detail. (Former NFU Official A, interview)

Views of this nature were replete in all the cases and highlight that the preferences of those who create lessons and those who implement them are strongly aligned in favour of single-loop lesson-learning that produces modest, but deliverable, formal-institutional outcomes. This tells us that there is a context-specific balance to be struck in an inquiry between easier to achieve single-loop lessons and potentially contentious, but perhaps more meaningful, double-loop reforms. In these inquiries, that tension was resolved by almost exclusively promoting the former, which is why we see little value-orientated learning in these cases.

Conclusion

The findings from all four cases show that these inquiries produced policy learning outcomes which reduced the likelihood of future failures. This principally occurred through the facilitation of two specific types of lesson-learning, which were ‘instrumental’ and ‘cognitive organisational’ in nature. These two types of policy learning delivered reforms which equipped future crisis managers with better policy tools and placed them within policy communities that enjoyed greater degrees of coordination.

These improvements meant that some of the policy failures which characterised past crises were not seen in subsequent events. Each inquiry was therefore something quite different from what a reading of the extant inquiry literature would have us believe. These were not ceremonial institutions. Neither were they agenda management tools. And most importantly, their reports were not ‘shelved’ by bureaucratic actors. Instead, they identified lessons, reforms were implemented, and crisis management performance improved.

There are of course caveats to be recognised. The data used here emerged from a small number of cases and a qualitative research design that privileged the interpretations of actors. It was also generated via a subset of public inquiries relating to crises, and it is certainly possible to hypothesise that inquiries that are not associated with traumatic events may not produce agenda-changing reforms. However, the value of this research does not emerge from the delivery of a definitive and generalizable answer to the question of whether or not all inquiries are effective policy learners. Instead it relates to the question mark that it places against what we think we know about these important institutions because this invites us to reconsider whether political science has got it right in relation to them.

In this regard, future research ought to address two questions. First, can the findings produced here be replicated in relation to different types of public inquiry that do not relate to crises? Answering this would take us towards a more generalizable statement about the public inquiry and one which is also capable of determining what these mechanisms are best placed to learn about. Second, are there other taken-for-granted aspects of our knowledge about the inquiry which may also be problematic? In this regard, assumptions about the ways in which political and bureaucratic elites supposedly steer and manipulate inquiries for their own purposes seem to warrant further investigation. Is there evidence for this widespread assumption and if not, how can we more accurately account for inquiries that do not get implemented? Regardless of the specific questions we return to, what we cannot do is continue to propagate assumptions about these institutions without meaningful data.