Keywords

Introduction

This chapter examines the approaches and strategies actually used by democratic governments in managing policy challenges and responding to complex and contested issues. Wicked problems are managed politically. Leaders may choose to avoid or downplay the tough issues, or they may redefine the issues to align with their existing agendas, and impose their own solutions. They may choose to take relatively familiar pathways of policy adjustment or, alternatively, seek new ways to tackle uncertainties and manage conflicts. The capacity of government leaders to achieve long-term improvement in a policy field depends on the quality of leadership, the capacity for stakeholder collaboration, the capacity to identify effective options and policy strategies, the availability of core skills and resources and the overall governance capacities of the system (Howlett, Ramesh & Wu, 2015).

The public policy literature has long speculated that some problems are too massive to be ‘tamed’ by government interventions and too broad-ranging and dynamic to be fully understood by scientific research. The debates of the 1970s about ‘big’ or ‘overloaded’ government in western democracies (King, 1975; Parsons, 1982) reflected neo-conservative demands to reverse the growth in government spending and halt policy ‘over-reach’. The neo-conservatives wanted to lower public expectations and reduce governmental ambitions to resolve enduring problems (such as inequalities and social unrest). Instead, governments should ‘go back to basics’, reduce the taxation burden of welfare state programs, and increasingly rely on non-state partners to address ongoing socio-economic concerns.

The public policy literature also highlighted the likelihood that governments will sometimes make incompetent or myopic policy choices (Hogwood & Peters, 1985), resulting in policy interventions that are not only ineffective but that might worsen the initial problems (King & Crewe, 2014). In other words, in some cases the policy ‘cure’ could be worse than the ‘disease’ (Logan & Preble, 2008; Sieber, 1981). Moreover, it is possible that a well-defined problem that is not inherently wicked might be drawn into wider intractable problem areas (a case of ‘wicked by design’) because political actors want to use the well-defined problem as ‘a surrogate to debate larger and more controversial problems’ (Nie, 2003, p. 314).

This chapter outlines the main policy governance strategies actually used by government leaders for dealing with wicked problems. The literature on this theme is surprisingly slim. Nancy Roberts (2000) identified three sets of strategies for responding to wicked problems, drawing largely from examples in international relations: competitive strategies (where power is dispersed but contested), collaborative strategies (where power is dispersed but not strongly contested) and authoritative strategies (where power is concentrated rather than dispersed). Her attempt to highlight the links between these strategies and the power dynamics within policy fields is instructive and useful. However, this threefold classification does not provide all the nuances required for understanding the diverse policy choices made by government leaders facing complex policy challenges. Hoppe (2010) has argued that the main dividing line in governmental responses to complex and controversial problems is between determining solutions either through ‘powering’ (i.e., impose the answer) or through a process of ‘puzzling’ (i.e., discuss the options). Again, this is a helpful insight into the spectrum of choices, but each of these two main strategic choices may include a wide range of categories reflecting the full suite of process strategies and institutional contexts.

In considering the available array of strategic choices, government leaders will not only assess the perceived threats, but also take account of their own political obligations (to their parties, stakeholders and supporters), and the institutional context of program legacies and resources they have inherited. They bring to the table their own leadership style and tactical preferences. Their personal style will be modified by the need to acknowledge the cultural perspectives and material interests of their own support base. As noted in Chapter 1, leaders are very actively involved in shaping perceptions of the problem itself, the context in which decisions need to be made, and the preferred responses. These persuasive efforts serve to legitimise a particular form of action preferred by the decision-maker. In effect, the problem-context itself is reconstructed as a ‘political arena’, in which leaders use persuasive mechanisms ‘to render situations more tractable and compliant to their own preferred form of authority’ (Grint, 2005, p. 1492). In considering policy priorities and actions, leaders also scan information about stakeholder opinions, or community debates about the nature and salience of problems. To reduce the possibility of failure, successful leaders review the likely reception of various policy options, examine the costs and benefits, and consider the implementation capacities of all the relevant governmental and non-state organisations that may be needed as partners (Howlett et al., 2015).

This chapter offers a broad classification of strategic process choices for policy responses, based on the actual observed behaviour of leaders (not to be confused with normative models that have been recommended for addressing specific policy challenges). The seven strategic process choices outlined below include problem avoidance, authoritative imposition, micro-management, science-based technocracy, incremental ‘muddling through’, collaborative dialogue and long-term coping and prevention strategies. Clearly these seven types should not be seen as stand-alone categories; they will tend to overlap in political and policy practice. In other words these various approaches are not neat alternatives but are usually found in combinations.

A full discussion of strategic approaches to policy governance would need to outline the various contexts in which the challenges arise, and the different capacities of leaders, governmental agencies and civil society organisations to cope with such challenges. Some crises and challenges are more ‘governable’ than others, as the effective levers available to government are always limited, and some ‘natural’ disasters are not avoidable through human intervention. There is no space in this book to examine detailed cases and circumstances, but Chapter 4 will broadly outline some of the different forms of ‘crisis’ which form the context of many contemporary wicked policy issues. Crises can be slow-moving, creeping and incremental, but with cumulative harmful impacts that eventually signal major risks and a need for action. Alternatively, crises can be acute, abrupt and highly disruptive in a more concentrated timeframe. In both the slow and fast scenarios, crises involve a diverse range of interconnected issues. They generally attract divergent perceptions about their scope, urgency and significance. These controversies arise under conditions of uncertainty about the knowledge base, the responsibilities for action, and the preferred forms of intervention. These differences and uncertainties confirm their status as wicked policy problems that eventually demand governmental attention and action.

Governmental Responses to Wicked Problems—Seven Strategies

Determining ‘fit-for-purpose’ strategies is always difficult, even for the most astute leaders and policy managers. The ‘best-available’ solutions crafted by actors in one situation are not readily transferable to other locations, because they are closely linked to their original political and institutional context. In those fortunate situations where robust solutions have been successfully negotiated, the agreed policies must be well implemented, raising a further series of governance and resource challenges (Crowley et al., 2020, Ch 8). Moreover, even the ‘best’ policies might have only short-term benefits, because economic and political circumstances may shift over time and policy goals might need adjustment. In some of the more complex and multi-layered policy problems, the standard for success might be very modest—a perspective that will be argued in more detail in a later section. For example, coping strategies to stabilise a problem and prevent harmful deterioration might be the best-available approach (albeit less than ideal) in some contexts or circumstances.

Seven major strategic pathways can be discerned when we observe how government leaders in democracies respond to wicked problems: avoidance and denial, authoritative coercive controls, micro-management of problem elements, technocratic problem-solving, incremental adjustment, stakeholder collaboration and ‘coping’ strategies. This classification highlights the processes chosen for steering or managing policy problems, rather than the specifics of policy design for different types of policy issues in various policy fields. And it is worth repeating that these seven types are not necessarily distinctive or self-contained choices; rather, they are likely to co-exist in policy practice.

Avoidance, Denial and Minimal Responsibility

A common response to complex problems and emerging threats is simply to deny their reality or significance. The capacity to ignore information and arguments, and the capacity to deflect attention from a potential issue, is an important form of power. These capacities have been well demonstrated in two fields of research—firstly, the literature on the impact of issue-framing and problem definition (Bacchi, 2009), as discussed in Chapter 1 above; and secondly, the literature on ‘non-decisions’, which can be seen as the capacity to block certain interests or issues arising, whether through deliberate veto behaviour or through the latent power of embedded routines and institutional practices (Hayward, 2006; Isaac, 1987; Lukes, 2005). This deliberate ignoring or denial of key information occurs in the private sector as well as governmental decision-making—such as the extended attempts by the tobacco industry to downplay and discredit scientific studies linking smoking and disease (Oreskes & Conway, 2010). Governmental and private sector actors can resort to ‘strategic ignorance’ by exaggerating uncertainties about an issue in order to avoid accountability (McGoey, 2019).

In recent years there has been heightened concern about the blatant disregard of ‘the facts’ by political leaders, whose partisan interests prevail over any considerations of objective analysis. It is often observed that politicians tend to reject evidence that contradicts their prior attitudes, regardless of whether they are populist leaders who articulate identity-based rhetoric or more accommodating politicians who are willing to discuss issues more openly. The scholarly research literature on ‘motivated reasoning’ has widely documented this phenomenon of biased cognition. Its specific application to the study of political debates has confirmed that inserting ‘more evidence’ into policy discussion does not moderate partisan bias of political actors (Baekgaard et al., 2019). Similar findings emerged in experimental research when citizens were asked to interpret key facts—the patterns of significance, attribution and blame were closely tied to partisanship (Bisgaard, 2019).

Successful democratic leaders seek to influence the content of policy agendas in particular ways, highlighting some issues and avoiding or minimising others. In doing so, they seek to influence public opinion, and they also seek to define the matters on which their own performance will be judged. In addition to limiting and prioritising the policy agenda, leaders are very concerned to avoid blame (Hood, 2002, 2011). One common tactic is to deflect responsibility for many of the issues raised in media debates, for example by arguing that the issue is really the responsibility of individuals, families, community groups, business investors, or another level of government. Leaders are also concerned to claim credit for positive outcomes on other related matters including the work of partners or delegates.

Denying the significance of a problem involves the exercise of persuasive power. Keeping a complex and intractable issue ‘off the agenda’ allows government leaders to avoid taking explicit action to address the issue (Shpaizman, 2020). Successful leaders shape how priority issues are understood and debated. The corollary is that leaders downplay or minimise the significance of issues raised by critical stakeholders, political opponents and the media. Information which casts doubt on the good intentions of government leaders are increasingly labelled as ‘fake news’, and an increasing flow of deliberate misinformation has become evident in political discussion (Iyengar & Massey, 2019; Persily & Tucker, 2020). To the extent that a significant issue remains a matter of public debate, leaders often argue that it is someone else’s problem.

However, wicked issues cannot simply be ‘wished away’. When government leaders lack the organisational capacity and political will to tackle these issues, they may nevertheless decide to take symbolic actions. In doing so, they might explicitly acknowledge the problem, but offer only a conspicuous gesture. Symbolic and weak policy responses have been termed ‘placebo policies’ (McConnell, 2020). Such interventions address the symptoms rather than tackle the underlying causes (e.g. of inequality, discrimination, aggression, etc.). A related political tactic is the intentional overlooking of information that could reflect badly on leaders and managers, akin to the notion of strategic amnesia in the literature on policymaking and policy learning (Stark & Head, 2019).

Intentional ‘under-reaction’ to a policy problem can be explained as driven by political leaders’ desire to avoid blame (Maor, 2021); but such responses can also be explained by ideological preferences and framing contests. Maor suggests that under-reactions may sometimes be counter-productive, in the sense that the issues could evolve into more intense forms that provoke political leaders into an equally inappropriate over-reaction (Maor, 2021). Other analysts who specialise in the study of stories and narratives have used the enticing metaphors of political dramaturgy and stagecraft. In this approach, government leaders tend to focus on managing ‘front-stage’ impressions (the official messages, as codified in speeches, media statements and policy documents), whereas the ‘back-stage’ complexities and contradictions experienced by citizens, stakeholders and frontline workers may be very different (Edelman, 1964; Hajer, 2009, p. 55; van Hulst, 2008; Schlenker & Pontari, 2000).

The concerns about ‘big’ or ‘overloaded’ government in the 1970s and 1980s were partly about cutbacks, but also about changing how public programs and services were delivered. The desire to reduce the size and the roles of the government sector led to various attempts to externalise responsibilities (away from state actors). This was pursued through three methods, all of which turned out to be double-edged. Firstly, governments delegated responsibility for operating utilities to non-state actors through privatising state-owned commercial businesses. Secondly, they outsourced service delivery programs through contractual agreements with non-state parties. Thirdly, they established ‘light-handed’ market mechanisms as an alternative to prescriptive regulation (Fink, 2011; Lobel, 2012; Salamon, 2002).

Privatisation immediately cut the size of the public sector, since private corporations replaced state-owned trading entities in such fields as water, energy, communications and transportation. However, in cases where the assets were monopoly businesses, such as utilities and airports, the state retained obligations to oversee fair treatment for consumers and potential competitors. The outsourcing of service delivery was achieved by competitive tendering to engage third parties (both private corporations and not-for-profit community organisations). However, while these non-state organisations delivered the public programs under contractual agreements, the state retained its obligations to fund the services, ‘steer’ the scope and conditions of service provision, and monitor compliance with the terms and conditions. Hence, while the service providers took formal responsibility for mistakes and potential fiascos, service design was largely a matter for the state and poor outcomes could reflect badly on the government designers. Finally, governments increasingly championed market mechanisms and voluntary codes of industry conduct as ‘light-handed’ approaches to influencing stakeholder behaviour, thus allowing businesses and consumers to adjust their own investment decisions or consumption choices. However, while avoiding the costly overheads associated with prescriptive rules and their enforcement (i.e. the twin burdens of ‘red tape’ and monitoring/policing of rules), the task of designing fair and efficient markets proved to be very difficult for state authorities.

In short, inaction and deliberate deferral of policy attention by government can be explained in several ways (McConnell & t’Hart, 2019). In some cases it is rational to ‘let the dust settle’, lower the temperature of disputation, or allow conflicting groups to articulate their values and interests before attempting mediation and conflict resolution (Cantekin, 2016). In other cases, government leaders seek to deflect attention from problems, minimise their responsibilities for issues, and attempt to depoliticise difficult challenges. Where they decide to take action, they often select symbolic actions and routine administrative techniques that would usually be used for ‘tame’, settled or manageable issues, but which do not work well for wicked problems. Alternatively, governments seek to manage at arm’s-length through contracts and markets. In most cases, the politics of avoidance, denial and minimisation amounts to the politics of distraction or the politics of managing ‘at a distance’.

Coercive Controls

The second type of response to an emerging problem is almost the opposite of avoidance and distancing; rather, the regime leaders relish the opportunity to grasp the challenge and to demonstrate they have the answers. They do not hesitate to impose policy solutions by using centralised and coercive approaches. This route is not confined to the leaders of authoritarian one-party regimes which seek to impose conformity and punish diversity of views. In multi-party political systems, government leaders might impose decisive solutions through executive action. This tactic is used to manage crises that are seen as requiring forceful and rapid responses. Populist leaders sometimes utilise a ‘heroic leader’ style of decision-making, especially on matters framed as security threats where emergency powers can be invoked to deal with external and internal threats. Real or supposed national security threats are often used to bolster support for the ruling party and to suppress dissenting voices in the name of patriotism and national identity (Wojczewski, 2020). As Edelman noted some decades ago:

….. there is always a ready audience for concerns about ‘national security’. Because such anxieties are easily aroused and because they can easily be directed against any domestic or foreign group that is labeled a threat, worries about national security are constantly evoked. (Edelman, 2001, p. 7)

In recent years, for example, the Hungarian government of prime minister Orbán responded to the refugee crisis, in defiance of the European Union policy framework, by imposing a range of measures to repel refugees; furthermore, in response to the pandemic crisis in 2020 he introduced emergency powers to suppress criticism. More generally the government has taken measures to harass and criminalise journalists, academics and civil society organisations that promote pluralism (Grzymala-Busse, 2019; Serhan, 2020), while also recently announcing plans to privatise the public universities.

National security concerns can be substantial and warranted; but centralised executive action by democratic leaders can proceed without also undermining democratic institutions and civil liberties. For example, research on Scandinavian governmental responses to domestic terrorism has shown that an effective response to terrorist incidents requires a high level of coordination between government agencies but does not require suppression of civil rights. Better coordination is widely seen as a necessary element in effective responses to many kinds of wicked problems (Kettl, 2003), but the political and managerial challenges of effective coordination can themselves be challenging (Christensen et al., 2013, 2016). Managerial coordination pressures have also been felt in emergency responses for natural disasters like fires, floods and storms. Emergency management organisations have strong command-and-control management styles; but they have recently recognised the need to design processes for ‘learning lessons’ and sharing experience, as a result of repeated natural disasters where previous knowledge about effective and coordinated responses was overlooked. For example:

Following major reviews ….there was recognition across the sector that many of the findings and recommendations made in these major reports were similar to findings made in earlier incident reports and reviews. The sector felt that many mistakes were being repeated. Despite an intention to continuously improve, there were no procedures or mechanisms to ensure opportunities to improve were implemented or shared. The [recent] increased level of collaboration between organisations provided an opportunity to develop a process to involve all organisations in a lessons management approach. (Victorian Government, 2020, p. 67)

In short, strong executive-led action is common in the face of disasters, emergencies and perceived security threats. Leaders play a large part in articulating the nature of the challenge and the type of response required. The politics of perceptions and framing are vital in this decision-making process. Implementation capacities and contingency planning are often overlooked, and lessons from past experience can rapidly be forgotten.

Compartmentalised Micro-Management

In practical politics, as well as in scientific inquiry, it is common for large problems to be analysed in small pieces rather than in their totality as complex systems. The intention is to identify bite-size chunks that can be carefully described and more easily managed. Complex wide-ranging problems can be intimidating and difficult to grasp. In policy analysis and practice, it is genuinely difficult to select the appropriate level of analysis and action. There is a strong tendency to focus on one visible symptom of a significant problem instead of searching to address the underlying causes. Moreover, real-world problems tend to be interlinked and ‘nested’, ranging from specific phenomena through to higher-level processes. In national and global terms, each of the 17 sustainable development goals of the United Nations occupies a huge terrain (see https://sustainabledevelopment.un.org/sdgs) and they all interconnect in various ways.

When summarising Rittel’s first seminar presentation on wicked problems in 1967, Churchman reported that it is common to undertake partial analyses by ‘carving off a piece of the [wicked] problem and finding a rational and feasible solution to this piece’ (Churchman 1967, p. B‐141), and then to ‘deceive’ people that the problem has been solved. As Rittel and Webber noted: ‘one should not try to cure symptoms: and therefore one should try to settle the problem on as high a level as possible’ (Rittel & Webber 1973, p. 165). From a similar perspective, Ackoff (1974) claimed that social and economic problems cannot be understood and addressed in isolation from their wider context:

Every problem interacts with other problems and is therefore part of a system of interrelated problems, a system of problems…..I choose to call such a system a mess…..The solution to a mess can seldom be obtained by independently solving each of the problems of which it is composed….Efforts to deal separately with such aspects of urban life as transportation, health, crime, and education seem to aggravate the total situation. (Ackoff, 1974, p 21; emphasis in original)

More recent research has argued that a complex system or a complex problem cannot be explained ‘merely by breaking it down to its component parts’; rather, the interdependent elements ‘interact with each other, share information, and combine to produce systemic behaviour’ (Cairney, 2012, p. 348). As noted in the following chapter, large complex problems take many different forms, each with its distinctive challenges for analysis and action. Instead of assuming similarities between problems, it is helpful to identify the distinctive configurations of each problem and ‘recognize how different problem properties can lead to different policy pitfalls’ (Ruhl & Salzman, 2010, pp. 99–100). It is important for government leaders and departmental units responsible for policy development to understand the dynamic causes of the problems, map their inter-relationships, and design appropriate policy frameworks that allow for adaptive management and effective implementation.

Despite the importance of understanding the deeper causes of policy problems and the interconnections between issues, democratic political regimes are generally geared to deal with problem elements rather than focus on whole systems of problems. The standard approach is to analyse specific components of a problem field in response to stakeholder pressure, and develop policy interventions at a targeted scale, to be measured through performance indicators for each project. Arguably there has been a shift towards a ‘proliferation’ of policy ‘projects’ in recent decades (Jensen et al., 2018; Sjöblom & Godenhjelm, 2009) as an alternative to designing integrated programs pitched at larger issues. For example, if the education system or the healthcare system are seen as deficient, the political solution is likely to be a series of projects to build new facilities in various locations, or perhaps a workforce initiative to train and hire more staff. If climate change increases the incidence of severe damage from floods and storms, the political solution might be to build stronger bridges and establish seawalls to mitigate coastal erosion.

Given the enormous scope and interconnected complexities of major policy issues, public debate on large and messy problems tends to lack careful definition and tends to amplify divergent viewpoints that are hard to reconcile. Therefore, careful attention to component parts of the problem could be a useful and attractive starting point. It may be practicable and productive to seek more consensus on narrower aspects of the problem. We noted in Chapter 2 that political ‘steering’ of policy debates is generally aimed at shifting messy unstructured problems into becoming more specific ‘well-structured’ problems. Community engagement and stakeholder participation might help identify matters where agreement can be fashioned and policies developed. But there remains a serious doubt about whether difficult issues can indeed be ‘solved’ simply through redefining them into ‘technically controllable’ issues (Hoppe, 2010, p. 88).

In short, there are widespread tendencies in the political and administrative systems to decompose or deconstruct a broad problem into more manageable constituent parts. The process of policy debate can confirm this tendency towards analytical fragmentation and compartmentalisation. This trend is both limiting and understandable. On the one hand, wicked problems at the macro level cannot be ‘tamed’ or ‘fixed’ by dissolving them into multiple elements, which are then tackled separately and independently. Such a strategy overlooks the requirement for understanding and addressing their systems context. On the other hand, incremental approaches are both useful and necessary, as will be seen in a later section below. There are many reasons to appreciate the contribution of small and cumulative changes. Small improvements can assist in learning about innovation through small-scale pilot schemes (see Chapter 7). Small wins (Weick, 1984) can inform the politics of change and can help to build momentum and support. Biting off small pieces of the problem can be seen as delivering tangible improvements, especially when the series of small measures is linked to strategies aiming for deeper change.

Technocratic Problem-Solving

The quest for rational and elegant solutions to problems, based on science and logic, has been a theme within liberal-democratic policymaking networks since the 1960s. Faith in the benefits of a scientific approach emerged as a counter to the perceived reliance on non-rational decision-making—i.e., politicised processes shaped by ignorance, prejudices, loyalties or traditions. The proponents of a more scientific approach argued that public policymaking should be based on the best-available knowledge concerning the policy challenges under consideration. In the 1980s and 1990s the concept of ‘evidence-based policy’ was elaborated by researchers and by government officials (Davies et al., 2000; Head, 2010a, 2015). The intent was that objective knowledge from the social sciences (as well as the health and environmental sciences) should have a prominent place in policy analysis and decision-making. Technology research (from the physical–chemical sciences) was already prominent in enabling the development of military weapons, computer systems and space travel; hence, perhaps it was possible for scientific research to also address the great social problems of social cohesion, equity and prosperity? In this context, champions of the social sciences highlighted the importance of rigorous evaluation of policy options, based on research into causal relationships and modelling the likely impacts of various policy choices. Evaluation of government programs would provide an important information base for policy improvement. Insights into the effectiveness of past programs would be complemented by detailed analysis of future policy options using quantitative measures of costs and benefits.

Thus, an ‘evidence-based’ approach emerged, advocating the production of rigorous and reliable knowledge, and promoting its utilisation within the policy process. Considerable efforts were made in many countries to invest in systematic data, analytical skills and evaluation guidelines that would provide foundations for a more evidence-based approach. Evidence-based policymaking (EBP) became an attractive ideal for professionals concerned with building robust information and improving the techniques for analysis and evaluation. However, EBP was also strongly contested from three related sources.

Firstly, EBP was criticised by participatory democrats for promoting a form of elitism that privileged the knowledge of ‘experts’ and legitimated ‘rule by technocrats’. In particular, critics claimed that when considering complex and contested issues, extensive community engagement was essential for clarifying acceptable and desirable outcomes (Fischer, 1990, 1993). Secondly, government ministers asserted their own form of elitism, by claiming that their ‘mandate to govern’ gave them the authority and the obligation to balance the claims of all stakeholders and knowledge-holders; and ministers necessarily made politically driven decisions for which they are electorally accountable. Governments might choose to recognise scientific/professional advice where it was useful for strengthening policy options favoured by government. As Michael Gove famously stated in the Brexit debate: ‘I think the people of this country have had enough of experts from organisations with acronyms saying that they know what is best and getting it consistently wrong…’ (see https://www.youtube.com/watch?v=GGgiGtJk7MA). Moreover, some of the appeal of populist politicians derives from the rhetorical contrast between their own championing of the people’s will and the power of hidden ‘elites’ that allegedly seek to rule. Thirdly, within the academic community, many researchers have suggested that complex problems cannot be fully understood solely through the lens of rigorous experimentation and statistical analysis. Many kinds of knowledge and experience are needed to understand a complex social problem and to assess how current programs and practices might be making a difference (Pawson, 2006). There are many relevant knowledge-holders, and it is important to encourage their interaction through networks; such networks are useful adjuncts to mainstream scientific inquiry (Ferlie et al., 2011). Competition and collaboration are both necessary in order to avoid one-sided thinking about policy puzzles.

In short, better information is now more widely available than ever before, as a result of investments in science and evaluation and its dissemination through informed commentaries. In principle, the quality of decision-making should have improved over time, owing to information and analysis about program performance. However, despite the flood of data, and diverse reports about evolving issues and risks, there remains widespread skepticism about institutional capacities to deal with emerging crises and complex social problems. Even if capacities for data gathering and analysis were to be improved, the ‘wicked problems’ perspective would suggest that some problems are too messy to be addressed satisfactorily through a scientific-expertise approach. Schön’s distinction between technical and contested issues is highly relevant. He contrasts manageable issues that are amenable to applied research on discrete topics, and complex situations that are messy and contested:

…there is a high, hard ground where practitioners can make effective use of research-based theory and technique, and there is a swampy lowland where situations are confusing ‘messes’ incapable of technical solution. The difficulty is that the problems of the high ground, however great their technical interest, are often relatively unimportant to clients or to the large society, while in the swamp are the problems of greatest human concern….There are those who choose the swampy lowland. They deliberately involve themselves in messy but crucially important problems and, when asked to describe their methods of inquiry, they speak of experience, trial and error, intuition, and muddling through. (Schön, 1983, pp. 42–43)

In summary, wicked problems cannot be well managed by technical experts alone. Wicked problems have to be managed politically, not just by scientific and professional experts. Government leaders cite research evidence when it suits their agendas, and they (occasionally) claim to be following expert advice on technical matters, such as decisions to impose unpopular restrictions to combat the spread of infectious diseases. In practice, experts provide advice but are rarely granted authority to make public decisions. Ministerial oversight and accountability generally prevail. Guarding against the possibility of technocratic decision-making by experts is rightly a concern in democracies, but the greater risk may be that elected leaders are making decisions based solely on appeals to political values or political self-interest. Some leaders have even encouraged the dismantling of evidence-based research capabilities in order to serve their partisan political interests (Boyle & Kotchen, 2018).

Incremental and Pragmatic Adjustment

Complex and difficult issues are regularly confronted in democratic political systems. But the manner in which these issues are framed, prioritised and managed can be diverse. The rhetoric of policy decision-making often uses the language of rational problem-solving and managerial effectiveness, thus implying a reliance on information and reasoned debate. But the policy process itself is fuzzy, political and conflictual. This situation has been explained in the academic literature by two main factors. One is the pluralist nature of group politics and public opinion in a democratic society; the existence of multiple interests and perspectives tends to make it difficult to achieve even a temporary consensus on goals and methods. The second factor is the limited capacity of decision-makers to deal with large volumes of information and diverse opinions across many issues, so that the decision-making system is characterised by ‘bounded’ or limited rationality (Forester, 1984). Accordingly, to use the terminology of Herbert Simon, the decisionmaking process may reach decisions that ‘satisfice’ the actors concerned, rather than achieve ‘optimised’ decisions via a comprehensive analysis of all the relevant information and policy options.

One of the major public policy frameworks, incrementalism, has drawn on these two factors. Charles Lindblom respected the importance of good information but he also argued that the rational-comprehensive picture of the policymaking process was deeply flawed. In this claim, he shared one apparent similarity with Rittel and Webber, but their reasoning was quite different. Rittel and Webber (1973) had criticised the data-driven rationalist model for its cognitive hubris and top-down character. Hence, while it promised comprehensive policy and planning capacity, the rationalist model could not actually develop a full understanding of wicked problems or chart effective pathways for their improvement. Lindblom, by contrast, began from an assumption that the US political system had pragmatically evolved in a piecemeal way to accommodate the imperfections and partialities of human understanding and the inevitability of group disagreements (Lindblom, 1959, 1979; Lindblom & Cohen, 1979). Lindblom argued that the policy-making process, with its many checks and balances and opportunities for correction, was essentially the politics of compromise and trade-offs. This amounted to the politics of ‘muddling through’, with mutual adjustment among competing interests:

Instead of reaching ‘solutions’ that can be judged by standards of rationality, policy making reaches settlements, reconciliations, adjustments and agreements that one can evaluate only inconclusively by such standards as fairness, acceptability, openness to reconsideration and responsiveness to a variety of interests. (Lindblom, 1980, p. 122)

Once policy decisions had been determined by the political executive and legislature, the process of implementing practical programs would then lead to further rounds of adjustment in a multi-layered polity involving many actors and organisations.

For Lindblom, rigorous policy analysis was seen as valuable, but rigorous analysis was more likely to be accessed by some actors rather than others. He noted that policy analysis was often commissioned and used for partisan purposes (rather than for objective evaluations and clarifications). Lindblom saw that the policy process included many activities where scientific rigor rubs up against power, interests and values. Policy understandings and policy designs were seen as shaped more by practical experience than science. Lindblom’s work became interpreted by some conservatives as endorsing the US system of incremental bargaining; conservatives wrongly discerned a normative argument in favour of incremental changes and rejecting major policy changes. In fact, Lindblom was sceptical about the likelihood of major change occurring, rather than supporting the conservative proposition that change was unwise. Lindblom’s argument was that ambitious policy reforms had to contend with the weight of institutional inertia and path dependency (Pal, 2011). Moreover, reform proposals could be blocked by powerful business lobbies or other vested interests, and those proposals that reached the legislature could be heavily modified or narrowed in the process of further debate and decision.

Incremental policy trajectories imply that the political system has stable institutional arrangements for considering and implementing policy. And to the extent that policy pathways become settled and institutionalised, they cease to be controversial ‘problems’ that attract priority attention. Over time, the programs that appear to offer effective methods for managing social problems may become absorbed or woven into the ‘settled’ arrangements of policy governance. Where the ongoing risks and challenges are seen as relatively routine, they are likely to be handled administratively with incremental fine-tuning and little discussion. Where the situation is more novel, adaptive management can generate incremental change with only moderate debate about the agreed underlying directions. At the other end of the spectrum, large and complex issues may generate serious questioning of problem framing and support for new paradigms. As Lindquist noted:

routine decision regimes focus on matching and adapting existing programs and repertoires to emerging conditions, but involve little debate on its logic and design .…; incremental decision-making deals with selective issues as they emerge, but does not deal comprehensively with all constituent issues associated with the policy domain; and fundamental decisions are relatively infrequent opportunities to re-think approaches to policy domains, whether as result of crisis, new governments, or policy spillovers. (Lindquist, 2001, p. 19)

The notion of ‘adaptive’ approaches to policy design and management has become very fashionable. Brunner (2010) suggests that ‘adaptive’ approaches to governance are consistent with making iterative adjustments and ‘learning by doing’ at a manageable scale. This is very different from attempting wholesale and comprehensive policy transformation:

adaptive governance suggests factoring the global problem into thousands of local problems, each of which is more tractable scientifically and politically than the global one. It also suggests harvesting experience from local communities and organizing them as networks to scale out and scale up what works in practice. (Brunner, 2010, p. 306)

A broader notion of adaptive management is also discussed in Chapter 4 below, in relation to governance approaches for the improved management of social-ecological systems.

Stakeholder Collaboration

Government leaders sometimes choose to tackle problems through a consultative approach incorporating stakeholder engagement and participation. The willingness of government leaders to work closely with business, community and professional groups is influenced by such factors as political leadership styles, the perceived capacities of stakeholder networks, and the nature of the policy challenges under consideration.

Government agencies often became more fragmented in the 1980s and 1990s, linked to the managerial efficiency and outsourcing agendas promoted under New Public Management (NPM). Coupled with a narrow faith in market-based solutions, managerialism undermined the capacity of government to address complex and wicked issues that required working in partnership with other sectors. The NPM approach reinforced governmental reliance on using contractors to deliver public services. But this contractualist approach was quite different from establishing collaborative partnership strategies for jointly addressing difficult social problems. Some governmental leaders became convinced that collaborative networks and forums should be strengthened to assist in discussing the nature of the problems, obtaining agreement on roles and responsibilities, and identifying a range of effective policy responses (Goldsmith & Eggers, 2004; Innes & Booher, 2010; O’Leary and Bingham, 2009). The advantages of collaboration were seen as improving knowledge about the issues, developing new options, and securing broader support for agreed new directions and strategies; the disadvantages were seen as the extended time and managerial effort required for dealing with the differing views and priorities of relevant groups, and the trap of ‘lowest-common-denominator’ outcomes (Ansell & Gash, 2008; O’Flynn et al., 2014).

When governments have used collaborative forums and networks to explore novel or difficult issues, senior officials are usually key participants who play major roles in funding the joint activities and chairing/convening the forum. Governments are reluctant to forego control of multi-stakeholder networks or venues that discuss topics impacting on the reputation of government leaders and agencies. On novel or emergent issues where the knowledge base is unclear and the challenges are unfamiliar, there is usually more support for gathering a wide range of perspectives and insights. But where the issues are long-standing and controversial, and the lines of conflict are well delineated, governmental leaders are more likely to support the establishment of a specialised public inquiry process which receives formal submissions and seeks a balanced future direction. In both cases, stakeholder participation is valuable for sharing knowledge of risks and exploring the consequences of various policy options. Stakeholder groups are likely to have concerns and disagreements about whether the chosen policy directions are both fair and effective, and on sensitive cultural issues (such as Indigenous knowledge and values), it is crucial to ensure that the benefits of new interventions are appropriately shared.

In the specific context of wicked problems, Xiang (2013, p. 2) has argued that the collective or ‘social’ nature of working with wicked problems and adaptation strategies requires a ‘holistic and process-oriented approach’ that is ‘adaptive, participatory and transdisciplinary’. Such an approach, he argues, would embody a learning and exploratory orientation. Ideally, this would help to reduce conflict and build trust, and ultimately produce better outcomes (Xiang, 2013, p. 2). This accords with Rittel and Webber’s argument that there is no ‘best’ solution to a wicked problem, but only provisional responses that are negotiable among relevant stakeholders. Conklin notes that because there are no clear and definitive solutions: ‘You don’t so much “solve” a wicked problem as you help stakeholders negotiate shared understanding and shared meaning about the problem and its possible solutions. The objective of the work is coherent action, not final solution’ (Conklin, 2006, p. 5). The assumption in much of the social science literature that collaborative methods could help to resolve a wide range of previously intractable problems generated a normative preference for collaboration. More recent work has paid more attention to the balance of costs and opportunities arising from working across boundaries in various contexts (Ansell & Gash, 2008).

There are several modes or levels of working together—networking, cooperating, coordinating and collaborating. None of these is intrinsically superior under all conditions. Rather, each may be better suited for specific tasks and challenges (Bryson et al., 2006). According to one experienced analyst, a strategy can be ‘appropriate for particular circumstances’, depending on the capacity of the actors to overcome the three standard limitations of working together—‘time, trust and turf’; and depending on their capacity to reach agreement about ‘a common vision, commitments to share power, and responsible and accountable actions’ (Himmelman, 1996, p. 27). The literature on the theory and practice of collaboration in public policy and service partnerships has lacked a coherent framework for designing and assessing effective collaborative arrangements (Bingham & O’Leary, 2006), but appropriate criteria for assessment are being developed (Emerson & Nabatchi, 2015; Sørensen & Torfing, 2021).

Skelcher and Sullivan (2008) make a case for taking a broad approach to the appraisal of cross-sectoral partnerships. They argue that collaborative performance should be assessed not only in terms of the ‘policy domain’ (i.e., achieving the desired policy outcomes), but also in terms of four additional dimensions. These are the ‘democratic domain’ (democratic performance, mainly about legitimacy); the ‘transformative domain’ (path-breaking behaviour, new benefits not otherwise possible without collaboration); the ‘coordination domain’ (mutually dependent exchange of resources); and the ‘political domain’ (generating high-level ideas that integrate the actions of divergent groups). In other words, achieving the benefits of improved environmental or social values is not necessarily the whole story. There are broader governance considerations concerning legitimacy and the quality of the change management processes.

Coping and Prevention Policies

Wicked problems are characterised by lack of agreement about the problem itself and about effective policy responses, as noted in previous chapters. There has been much discussion about how to overcome these disagreements either through better science or better politics. One route has been to improve the knowledge base, in the hope that deeper understanding of causal factors can generate better understanding about policy improvements. Another route has been to improve stakeholder engagement and processes for brokering political compromises. But what if the availability of an effective solution is doubtful? What if the problem is deeply embedded in social norms and behaviour, and thus not amenable to quick solutions or remedies with lasting benefits? Examples might include the challenges of gender inequalities, ethnic or religious discrimination, health inequalities, domestic or family violence, the abuse or neglect of children, widespread use of illicit drugs and patterns of entrenched poverty found in particular localities or kinship groups. A series of small initiatives may benefit some vulnerable groups, but tackling large problems through a long-term strategic approach may require a new paradigm that can unite most of the stakeholders and elevate the sense of policy purpose.

Let us imagine that political leaders commit to achieving long-term improvement in one of these wicked problem fields. Following the initial stage of raising awareness and announcing broad reform objectives, early measures might include building a broad-based coalition of support for stabilising the problem and preventing further deterioration. Discussion with stakeholders might underline the need for a long-term strategy with many stages, drawing on the resources and experience of many sectors. Such a strategy would encompass overlapping layers of activity and a broad mix of policy instruments. It would recognise the complex interaction between social, economic, health, educational and other factors, rather than presume there is a single or fundamental root cause that explains all the phenomena. This strategy would pursue practical changes on specific matters (e.g. provide support services for victims of violence), but the strategy would also acknowledge that the achievement of substantial ‘systemic’ changes take a considerable time to materialise. Tackling the ‘upstream’ or underlying causes of harm is the basis of ‘prevention’ policies. The term prevention should not be confused with a bold claim to suppress or eliminate the problem, like the use of a powerful vaccine. ‘Prevention’ approaches in complex social fields are about reducing the probability of avoidable harms, through tackling underlying causes and building the skills needed by vulnerable groups to take protective action. For wicked problems, there are no magical solutions (though some programs are clearly better than others). Prevention is a policy paradigm that directs our attention to underlying causes of potential harm and the need for concerted action.

For example, the international literature on public health focuses on ‘prevention of disease and health promotion’, in order to avoid the greater suffering and expense of treating diseases through hospital services (UK-DHSC, 2018; US-CDC, 2021). Well-known examples of health promotion campaigns include information about the harms caused by air pollution, alcohol, smoking and obesity, together with positive advice about healthy lifestyle choices and sound hygiene practices to combat communicable diseases. Screening programs can facilitate early detection of health problems and more successful remediation. ‘Prevention’ can be seen as a long-term approach for managing and improving embedded problems, and some examples from social policy, criminology, public health and environment are discussed in later chapters. Advocates of prevention urge that precautionary actions should be taken to reduce the probability of large-scale problems developing.

As outlined by Ian Gough:

The case for preventive public policy is ever present in large welfare states. The debates over health, crime, early years interventions and many other areas of social policy stress the advantages of prevention over coping, cure, compensation or confinement. This emphasis has been motivated by a combination of normative and economic reasons: it is better for human well-being to prevent harm than to deal with its consequences. (Gough, 2015, p. 307)

Prevention approaches usually distinguish between three levels, with most attention at the first and second levels. ‘Primary’ prevention programs are directed at the whole society by addressing potential risk factors (those which cause harm) and by building capability factors (which protect against harm). For example, school education and community health services are designed to be widely accessible, assuming public resources are available. ‘Secondary’ prevention focuses on targeted early interventions to address the early stages of a harmful condition among identified social groups at higher risk of harm; while ‘tertiary’ prevention overlaps with more intensive service systems that tackle the impact of more serious conditions.

For example, governments have sought to mitigate the challenge of domestic terrorists who are motivated by extremist ideological doctrines. A standard crisis response by the security forces (police, military, cyber) has been to defend critical infrastructure facilities and communications systems, and to visibly occupy and protect public spaces. However, other responses include educative programs targeting ‘at-risk’ groups, e.g., intensive training courses to encourage the rehabilitation of low-level offenders, and education programs to influence the attitudes of social groups that are believed to be most vulnerable to the ideological messages of terrorist networks (Fischbacher-Smith, 2016). In designing these programs to counter violent extremism, program managers are likely to use public health prevention approaches. These measures are likely to be qualitative, educative, and take account of complex issues around social identity and social interaction rather than policing and coercion (Weine et al., 2017).

The advocates of preventive public policies claim that education services and early intervention programs are cheaper (and more effective and more humane) than responding to the acute harm generated by full-blown social crises, which are obviously very costly and difficult to treat. The core purpose of preventive programs is to avoid deterioration on key problem indicators—social, health, economic, environmental, etc.—and to provide a strong platform for future improvement through targeted initiatives. Examples include policies to tackle addiction, reduce harmful consumption, provide skills and training, mitigate crime and anti-social behaviour, design and enforce industry standards for environmental health, provide social services to diminish child abuse, and fund programs to counter deep and persistent disadvantages suffered by children and families.

Prevention approaches make use of rigorous data analysis and multi-factor causal modelling to understand complex social patterns and especially the risk factors that exacerbate social problems; indeed, many proponents of ‘prevention science’ are strong advocates of hard-science approaches and experimental trials (Baron, 2018; Boruch & Rui, 2008; Campbell Collaboration, 2021; Kellam & Langevin, 2003). However, it can equally be argued that preventive programs are less about producing technocratic solutions and more about influencing complex social interactions. In this sense, prevention approaches seek to utilise the best-available knowledge but they are more deeply aligned with ‘resilience’ and ‘coping’ strategies than with rational-comprehensive planning. The resilience and coping dimensions are further discussed in later chapters.