Many scholars see a close connection between the discussion of wicked problems and the analysis of complex systems (Byrne & Callaghan, 2014; Geyer & Cairney, 2015; Geyer & Rihani, 2010; Ney, 2009; Tiesman et al., 2009). The policy literature on ‘complex systems’ is growing rapidly. The OECD has endorsed the value of exploring ‘how systems approaches can be used in the public sector to solve complex or “wicked” problems’ (OECD 2017, p. 4). Its recent report on systems thinking states that seeking to change the ‘dynamics of a well-established and complex system requires not only a new way of examining problems, but also bold decision-making that fundamentally challenges public sector institutions’ (OECD 2017, p. 3).
Democratic governance is inherently challenging, marked by political competition between actors with divergent perspectives and priorities. Democratic governance is full of trade-offs and paradoxes (e.g. reconciling stability and change, effectiveness and legitimacy, efficiency and fairness, state and non-state responsibilities for action). Given these complexities and uncertainties, broad capacities for strategic policy design, inter-group negotiation and collaborative implementation are crucial. The simplifications of ‘managerialist’ decision-making and target-driven efficiency regimes are viewed with suspicion by complexity thinkers (Ansell & Geyer, 2017; Eppel & Rhodes, 2018; Room, 2011; Seddon, 2008). Colander and Kupers (2014) draw a sharp contrast between two conceptions of governmental policymaking and knowledge. The rational approach focuses on policy instruments that predict and control outcomes, whereas the complexity approach highlights multiple interactions and perspectives:
The policy metaphor in the complexity frame changes from an image of government behind the steering wheel driving on a well-lit road, to an image of government trying to drive a car, with the windshield covered in mud, going down an unlit, winding road, with hundreds of people grabbing the wheel. (Colander & Kupers, 2014, p. 26)
Complexity theory, originally developed in the bio-physical sciences, draws attention to the multiple interconnections, feedback loops and surprising side-effects that can often undermine the aspirations of leaders to ‘control’ their socio-political systems. The term ‘complex’ is intended to signal the organic and interactive aspects of systems, rather than the mechanical aggregation of elements or components. For example, Foden (2018) playfully emphasises the difference between (1) understanding and influencing the behaviour of a sophisticated living organism, such as a cat, and (2) understanding and influencing the operating mechanism of a simple machine, such as a clock. Peter Senge (2006, Chapter 6 and Appendix 2) talks about the difficulties of understanding and responding to systems, and the typical errors and failings that arise from these misapprehensions.
A core tenet is that a ‘system’ understanding is needed to understand how the structure of the system influences the behaviour of the system. This is different from the standard approach of focusing on events/issues (e.g. an algal bloom) or a trend (e.g. increasing nutrient levels in a river basin), which are a typical focus of policy action but may not tackle the underlying structural issues. Complexity analysts in public management and public policy have argued that the modern era is marked by crises and rapid changes that ‘cascade’ across borders and across policy domains. These problems are so serious that they ‘challenge the steering capacity of governance’ (Duit & Galaz, 2008, p. 311). Accordingly, analysts need to research ‘the problem-solving capacity of existing multilevel governance systems in the face of change characterized by nonlinear dynamics, threshold effects, and limited predictability’ (Duit & Galaz, 2008, p. 329).
In policy analysis for health systems, Glouberman and Zimmerman (2002) distinguish between the ‘complicated’ technical and managerial knowledge required for managing modern health service systems (based on applying professional expertise to address known challenges) and the more ‘complex’ and contestable aspects of designing healthcare systems. The ‘complexity’ dimension in this example denotes the disagreements on values, ideologies, priorities and partner responsibilities. Such disagreements in healthcare can occasionally lead to policy gridlock and polarisation concerning the design of service systems and the balance of public/private roles. The literature on public health policy and services has further elaborated on the theme of ‘complex systems’ analysis (e.g., Carey et al., 2015; Hawe, 2015; Peters, 2014) in order to make sense of the interactions between various levels of activity, multiple actors and conflicting goals in healthcare. A systems approach can make use of both qualitative and quantitative information to construct models of how various factors and proposed actions may interact to produce not only desired effects but also unintended effects (Hawe, 2015; Haynes et al., 2020; Lich et al., 2013).
The field of environmental policy research (further detailed in Chapter 5) has also embraced complex systems analysis, including extensive work on the management of natural resources, ecological systems and climate change responses. Systems thinking emphasises the need for interaction and discussion among experts and stakeholders to map the dimensions of risk and disruption, and to consider a range of pathways to address these risks (Ison & Straw, 2020). The tendency for decision-makers to demand knowledge certainty might privilege calculative methods such as cost/benefit analysis rather than broader combinations of knowledge and experience that can better deal with the complex risks and uncertainties of emerging issues (Stirling, 2010).
The literature on complexity in public administration has also demonstrated that good coordination and planning are crucial (Christensen et al., 2016; Kettl, 2003; Pierre & Peters, 2005) for ensuring that responsibilities and priorities are clarified, implementation activities are well-resourced, monitored and adjusted (Kiel, 1994), and that governance networks and stakeholder participation are well managed (Koliba et al., 2011). This approach is broadly consistent with the ecological literature on ‘adaptive governance’ (Chaffin et al., 2014), which highlights the importance of capacities to respond to the inevitable shifts occurring within complex systems. Traditional process-oriented management of projects and programs is regarded as too inflexible and bureaucratic to address the unpredicted and unintended outcomes of system changes and to engage with ripple effects and spill-overs. Adaptive leadership is needed to renegotiate the trade-offs among policy objectives and stakeholder interests, while preserving the governance legitimacy built through shared goals. Similar themes have emerged in research on the multiple entangled issues inherent in the policy and planning regimes of large cities. The findings in the classical studies by Pressman and Wildavsky (1973) and by Rittel and Webber (1973) have been elaborated in many recent studies. For example, Karen Christensen found that complexities abound in the interactions between layers of city government and industry sectors, and across several types of issues. She also detected several forms of innovation and learning in city planning (Christensen, 1999, p. 96).
What are the knowledge challenges of coping with complexity? Some forms of complex problems (such as designing and building infrastructure) can be addressed through high levels of coordination, managerial efficiency, technical skills and sufficient funding, whereas other forms of complexity are characterised by high levels of knowledge uncertainty and disagreement about objectives. Reliable knowledge about future trends and social disruptions is seldom available, especially when some of the causal factors underlying a major crisis are outside the control of national decision-makers. Gaps in knowledge are normal; and attempts to fill these gaps are constantly being undertaken by both scientists and practitioners. However, the organisational context is also very important. Emery Roe’s analysis of the reliability challenges facing the managers of infrastructure enterprises (water, energy, transport) found that there were three forms of unpredictability: (1) low risk with key factors known and controllable; (2) some uncertainties where the likelihood or impact of reliability failure are not known; and (3) high uncertainty (many unknowns) regarding both likelihood and impacts. Roe argues that these situations correspond with three operating styles—controlling, adaptively managing, and coping with instabilities (Roe, 2020 p. 76). Roe argues that more attention should be given to situations where ‘control’ is not possible and where uncertainties have to be the focus of networked discussion.
Management consultants in the 1990s developed a classification of knowledge adequacy. Knowledge that is regarded as robust and relatively complete is described as the field of ‘known knowns’. More relevant to wicked problems, knowledge gaps that have been identified in key areas of concern constitute the field of ‘known unknowns’; these become a major focus for scientific research and practitioner learning. Beyond the comfort zone of existing reliable knowledge and calculation lies the sphere of ‘unknown unknowns’, understood as a realm of radical uncertainty, and a massive challenge for scientists, practitioners and decision-makers. Donald Rumsfeld, when US Secretary of State, focused on identifying ‘unknowns’ relevant to potential hostile threats to US security. Rumsfeld noted that it was very difficult to anticipate such threats, and to plan for appropriate and timely responses, in the absence of clear and abundant evidence (Rumsfeld, 2011, p. xiv). In foreign policy decision-making by the White House, especially after the 9/11 terrorist strike against the World Trade Center in New York, this preoccupation with the challenges of ‘unknown’ security threats became transformed into the doctrine of ‘pre-emptive’ action, whose primary goal was to ensure that hostile states and terrorist networks did not access weapons of mass destruction (Dershowitz, 2006, Ch 5; Suskind, 2006, pp. 62, 150). Pre-emptive action against terrorism has been described as especially risky and contentious (Stern & Wiener, 2006).
However, the urgent and well-funded responses to military security challenges can be contrasted with the weaker governmental responses to several other high-impact policy challenges. For example, in the field of US climate change policy, with its powerful corporate stakeholders and slow timelines, there has been a notable absence of precautionary and pre-emptive actions backed by well-funded programs. A high sense of urgency has not been strongly conveyed to the political executive in many countries by core stakeholders through public debate and community pressures. Debates about problem framing are highly relevant for complex issues. For example, ‘climate change’ can be framed as a scientific issue with expert technical solutions, or as an economic livelihoods issue where the policy response could focus on creating new jobs. Heather Cann (2021) shows that a climate and energy policy reform package was endorsed by the Illinois legislature owing to its economic orientation.
The character of each wicked problem may be very hard to discern, owing to complexity, uncertainty and divergent perspectives. Renn and colleagues have argued that ‘risk governance’ at a system level is fundamentally shaped by the need to deal with complexity, uncertainty and ambiguity (Renn et al., 2011). Conventional research projects, whether basic or applied, are generally targeted at filling known gaps in knowledge. Knowledge is generally patchy, and research aims to accumulate more information in order to tackle the ‘known unknowns’ (Pawson et al., 2011). However, the disruptions and surprises associated with new crises can raise concerns about the radical uncertainties of ‘unknown unknowns’. What kind of knowledge and expertise are needed in situations where—as Funtowicz and Ravetz (1993, 2003) remind us—facts are uncertain, values are disputed, stakes are high, and decisions are urgent? In this ‘post-normal’ challenge to the knowledge base for policy action, conventional scientific methods of data collection and analysis are seen as too narrow and too slow to grasp the complexities of rapid change. Network-based forums of diverse experts and stakeholders are recommended to examine the implications of possible future scenarios under conditions of uncertainty (Gerlak et al., 2021). Strategies for coping with uncertainty and turbulence ‘should not be about predicting the future—which is unpredictable by definition—but about devising methods and systems for handling the unexpected’ (Grint, 1997, p. 64). Aven and Renn (2010) have argued that different processes are needed for eliciting relevant knowledge for risk analysis and policy response, depending on the nature of the knowledge challenge—for example, is it a problem of causal complexity, uncertainty about impacts, ambiguity arising from value differences, or all of these together? (Aven & Renn, 2010, p. 187).
Policy analysts have outlined the processes by which problems are debated and ‘managed’ until they reach a provisional conclusion. Relatively simple or ‘tame’ issues with agreed solutions tend to settle into routine patterns of administration, in which incremental adjustment and performance monitoring are accepted as the dominant modes of governance. But sometimes the underlying issues may not be permanently resolved through policies decided and programs implemented. Contests about problem framing and prioritisation are recurrent. Given the complexities arising from various forms of stakeholder conflict and knowledge uncertainty, the policy governance challenge is to identify processes for dealing with the messy, ambiguous, controversial or unstructured nature of wicked problems. This policy governance challenge is exacerbated by the fact that the political theatre of public debate tends to focus on the more controversial and emergent policy problems—those which tend to require ‘non-routine’ processes to help identify long-term policy responses that are effective, feasible and legitimate. Serious gaps in knowledge and understanding may increase the likelihood that policy decisions about crises and wicked problems will be based primarily on leaders’ intuitions or political ideologies. This is especially risky for policy crises on high-stakes issues. The different types of crisis situations and different policy responses are discussed in the following section of this chapter.