Appeals to evidence for the resolution of wicked problems: the origins and mechanisms of evidentiary bias

Abstract

Wicked policy problems are often said to be characterized by their ‘intractability’, whereby appeals to evidence are unable to provide policy resolution. Advocates for ‘Evidence Based Policy’ (EBP) often lament these situations as representing the misuse of evidence for strategic ends, while critical policy studies authors counter that policy decisions are fundamentally about competing values, with the (blind) embrace of technical evidence depoliticizing political decisions. This paper aims to help resolve these conflicts and, in doing so, consider how to address this particular feature of problem wickedness. Specifically the paper delineates two forms of evidentiary bias that drive intractability, each of which is reflected by contrasting positions in the EBP debates: ‘technical bias’—referring to invalid uses of evidence; and ‘issue bias’—referring to how pieces of evidence direct policy agendas to particular concerns. Drawing on the fields of policy studies and cognitive psychology, the paper explores the ways in which competing interests and values manifest in these forms of bias, and shape evidence utilization through different mechanisms. The paper presents a conceptual framework reflecting on how the nature of policy problems in terms of their complexity, contestation, and polarization can help identify the potential origins and mechanisms of evidentiary bias leading to intractability in some wicked policy debates. The discussion reflects on how a better understanding about such mechanisms could inform future work to mitigate or overcome such intractability.

Introduction: wicked problems and ‘evidence based policy’

Advocates for ‘Evidence Based Policy’ (EBP) often present it as an idea which can resolve the political challenges of decision making, serving to make policies more effective and efficient (c.f. Coalition for Evidence-Based Policy 2015; UK Government 2013). EBP is conceptualized as a source of agreement or political convergence—an irreproachable goal which places rationality above politics—and where “pragmatism replaces ideology”(Solesbury 2001, p. 6). The ability for evidence to ‘rationalize’ policy making, or to overcome political debates, is therefore particularly questionable when considering so-called ‘wicked’ problems of social policy. In their seminal 1973 work in this journal, Rittel and Weber began with a powerful assertion that “[t]he search for scientific bases to confront social policies is bound to fail” (p. 155). The authors particularly judge social problems as wicked due to key differences in the nature of social phenomenon to those studied by natural science. Natural science problems are said to be definable and separable, with solutions that are ‘findable’—which the authors contend are not the case for almost all questions of social planning.

In recent years, the literature on wicked problems seems to be increasing exponentially. Indeed, a simple search on Web of Science ¹ for the term “wicked problem*” brings up just 11 hits between 1990 and 1999, 109 hits between 2000 and 2009, and 391 hits between 2010 and 2015. Wicked policy issues have also been defined as having high levels of contestation between opposed groups (what Rittel and Webber refer to as a lack of an undisputable public good), as well as technical challenges to being solved (B. W. Head 2008; Roberts 2000). Other features with direct relevance to concepts of EBP includes a lack of scientific consensus due to uncertainty or complexity in the nature of the problem (Head 2008; Koppenjan and Klijn 2004).

The Australian Public Service Commission (2007) has similarly pulled out a list of seemingly relevant features of wicked problems, as follows:

• They are difficult to define;

• They have interdependencies or are multi-causal;

• Attempts to address them may have unforeseen consequences;

• The problems are often not stable;

• They have no clear solutions;

• They are socially complex;

• They do not sit within the responsibility of a single organization;

• They involve changing behavior;

• They are characterized by chronic policy failure.

A challenge with this growing number of characteristics attributed to wicked problems, however, is that if the concept of wickedness becomes too broad, or too all encompassing, then it can reduce its analytical usefulness. In the last year or so alone, the term been applied to a wide range of issues cutting across a range of policy sectors—such as housing and urban planning (Walter and Holbrook 2015), euthanasia policy (Raisio and Vartiainen 2015), prison education (Czerniawski 2016), tobacco use (Rigotti and Wallace 2015), amongst a host of others—expanding to virtually all public policy concerns (not only limited to classic social policy concerns of the welfare state or provision of social services). So many issues and features of design issues can be found to contain elements of wickedness, however, that Coyne (2005) has gone so far as to argue that “[w]ickedness is the norm. It is tame formulations of professional analysis that stand out as a deviation (p. 12).” Indeed, the term ‘super-wicked’ problem has also been developed (c.f. Varone et al. 2013; Levin et al. 2012; Lazarus 2008), to potentially regain some analytical utility through a narrower focus on additional concerns. The term has particularly been applied to environmental policy or climate change specifically, with Levin et al. (2012) explaining how these issues have additional ‘wicked’ features such as temporal urgency, or solutions required from those causing the problems.

One way to potentially regain some analytical use of the concept of wickedness, however, can be to reflect on those aspects of wicked problems that present challenges to current calls to achieving ‘evidence-based policymaking’, and to reflect on whether there are ways to address some of these challenges. The fact that wicked problems are multi-causal, or have unforeseen consequences, for example, raises fundamental questions about which bodies of evidence are relevant, or how to prioritize between bodies of evidence—questions many evidence advocates have not addressed yet, often simply deferring to technical rigor as a way to judge evidence rather than policy relevance (Parkhurst and Abeysinghe 2016; Booth 2010). Some authors have adopted the language of ‘evidence informed policy’, however, (c.f. Hawkins and Parkhurst 2015; Brown 2015, p. 5743; Oxman et al. 2009) to chart a middle ground position that helps to address some of these challenges by particularly allowing explicit reflection on the fact that multiple social concerns are relevant to decisions.

However, even recognizing the fundamental importance of how policy problems involve multiple concerns, wicked problems may continue to challenge the assumptions of evidence advocates that robust and rigorous evidence can serve as a source of policy agreement, or provide a means to overcome ideological differences. Of particular relevance is where wicked problems are said to be characterized by ‘intractability’, with intractable controversies described by Schön and Rein (1994) as “highly resistant to resolution by appeal to evidence, research, or reasoned argument” (p. xi). The authors further explain that: “by focusing our attention on different facts and by interpreting the same facts in different ways, we have a remarkable ability, when we are embroiled in a controversy, to dismiss the evidence adduced by our antagonists” (p. 5). Yet the fact that appeals to evidence may not be able to solve political debates appears to be excluded from much contemporary EBP discourse, which continues to perpetuate the idea that a rational evaluation of evidence of ‘what works’ can solve policy problems in fields such as education, criminal justice, or international development (c.f. Davies 1999; Slavin 2008; MacKenzie 2000; Rittel and Webber 1973; International Initiative for Impact Evaluation 2009).

Evidence advocates voice continued frustration when evidence fails to be used as expected or desired. They raise particular concerned over what is seen as the political ‘misuse’ of evidence, lamenting so called ‘policy based evidence making’ instead of ‘evidence based policy making’; e.g. cases where evidence is strategically selected, manipulated, or misrepresented to pursue political goals (Marmot 2004; Boden and Epstein 2006; Guenther et al. 2010). In contrast, ‘hierarchies of evidence’ are promoted that emphasize methodological approaches such as randomized controlled trials (RCTs) or systematic reviews to provide relevant evidence in supposedly unbiased ideal ways (Haynes et al. 2012; Chalmers 2003). Yet critical policy authors have countered that there is a need to move beyond a ‘naïve rationality’ (Russell et al. 2008) that assumes evidence is somehow outside politics. Instead, these authors point to the reality that public policy decisions fundamentally involve choices between competing social values and competition between interest groups (Head 2010; Lin 2003; Greenhalgh and Wieringa 2011; Russell et al. 2008). Appeals to evidence that particularly recommend policymaking be guided by evidence hierarchies have been further critiqued as imposing de facto political priority to those issues where research has already been conducted, or which are conducive to evaluation via experimentation (Barnes and Parkhurst 2014; Parkhurst and Abeysinghe 2016).

In many ways, the divisions between champions of EBP and critical policy scholars reflect deeper epistemological divisions between positivists and post-positivist or constructivist thinkers. The language of EBP often reflects what might be described as a positivist assumption that the world is knowable and problems fundamentally solvable, while those from a more constructivist tradition would argue that all problems are ‘problematized’ rather than having any inherent or natural questions within them (c.f. Bacchi 2009). Without attempting to resolve these tensions, this paper can be seen to take a critical yet pragmatic approach. In this paper I do start by acknowledging the existence of many established policy problems and existing debates. The paper is critical, though, in that it recognizes there can be contestation about what is considered relevant within a policy debate—thus affecting which bodies of evidence are perceived to be relevant in the first place. However, this recognition does not mean it is impossible to assess the technical validity of pieces of evidence (or the validity of the use of pieces of evidence) either. Indeed, this paper aims to focus particularly on these two issues—which evidence is deemed relevant, and the scientific validity of evidence use at hand within the parameters of a given policy debate.

As such, in this paper I aim to explore features of policy problems that appear to manifest in intractability, which in turn may perpetuate problem wickedness by undermining efforts to resolve those problems through evidence utilization (even in cases where competing groups would both otherwise accept a need and a use for relevant evidence). To do so, however, it is first necessary to distinguish two distinct features of ‘bias’ in the use of evidence that undermines the EBP rational ideal and can drive many instances of intractability: ‘technical bias’—referring to invalid uses of evidence; and ‘issue bias’—referring to how pieces of evidence direct policy agendas to particular concerns. Drawing on the fields of policy studies and cognitive psychology, this paper explores the ways in which competing interests and values manifest in these forms of bias, and shape evidence utilization through different mechanisms. Based on these reflections, a conceptual framework is presented reflecting on how the nature of policy problems in terms of their complexity, contestation, and polarization link to the potential origins and mechanisms of evidentiary bias that may lead to intractability in some wicked policy debates.

Two forms of bias

It is hypothesized here that overcoming (or avoiding) policy intractability can serve as an important goal of the policy sciences in their search for solutions to social problems. Within such efforts, evidence is obviously critical to inform decisions, but from both perspectives of evidence advocates and critical policy scholars, there are expressed concerns over the politics of evidence. For those who embrace EBP, there are worries over political interests driving the distortion or selective inclusion of evidence, leading to invalid conclusions or ineffective interventions. For those critical of the EPB movement, a key concern is over the depoliticization of policy making—that we see social values obscured in appeals to evidence, and where certain types of evidence are given priority because of their perceived methodological rigor, rather than their policy importance or relevance.

In a forthcoming monograph (Parkhurst 2017, in press), these concerns have been described as forms of evidentiary bias, and both can be understood here as important in how they link to the policy intractability of some wicked policy debates which do not seem to be able to be resolved through appeals to evidence. The first form of evidentiary bias can be defined as technical bias—which refers to situations where pieces of evidence utilized would be judged as scientifically invalid, or where favorable findings are cherry picked from a larger body of relevant evidence, rather than reviewed rigorously. The second form of bias, however, is defined here as issue bias. As policy decisions involve multiple concerns, each of which may have its own relevant evidence base, the decision to include one or another body of evidence, or indeed the decision to create evidence on one or another issue, will inevitably prioritize certain concerns over others (biasing the decision to particular issues). Policy studies audiences are well aware of this, and indeed would not directly see the choice of social values through democratic processes as a problem; but it is the obfuscation of those values, or the indirect or implicit imposition of them through particular uses of evidence, which raise concern over issue bias.

On initial inspection, it may seem that issue bias better captures features of problem wickedness, by reflecting the competing goals or interests of policy actors. Technical bias may further appear ‘tamer’ than its counterpart, given that, in theory, appeals to scientific best practice or inclusion of all relevant evidence may render a solution to technical errors. Yet the argument is not being made here that these forms of bias themselves are features of wickedness. Rather the point is that both technical bias and issue bias can drive intractability; and it is intractability which can at times be a feature of wicked debates; perpetuating a lack of policy solutions emerging. The potential to solve or mitigate some forms of bias more than others is indeed important, however, and points to areas where there may be opportunity to reduce intractability. Achieving this, however, first requires greater delineation of both forms of bias, followed by exploration of their origins and the mechanisms by which they manifest in political arenas.

Technical bias

There is, of course, a rich literature mapping out a variety of ways that research evidence can inform policy making [c.f. the classic work of Weiss (1979) identifying seven meanings of ‘research utilization’ or more recent discussion on evidence use by Nutley et al. (2007)]. While some argue that modern EBP advocates too-narrowly focus on particular forms of research use (such as Weiss’ ‘knowledge driven’ or ‘problem-solving’ models) in their assumption that evidence can solve policy problems, there are still some valid concerns about evidence manipulation voiced by EBP advocates. Pieces of evidence can indeed be interpreted in ways that are more or less faithful to their methods and results, and bodies of evidence can be reviewed more or less comprehensively. Deliberate manipulation of findings is a particular problem, but also problematic for informing policy are cases of selective cherry-picking of evidence, as well as interpretations of research findings that are not valid with respect to the methodology or data of the study. All of these examples provide cases of technical bias, which can be defined from a perspective of scientific best practice. Yet technical bias can take many forms. Indeed, empirical examples illustrate that such bias can occur in either the creation, selection or the interpretation of evidence.

Technical bias in the creation of evidence

At times evidence is biased simply because research is conducted in ways that undermine good scientific practice. Some of the most obvious examples of biased evidence creation can be seen where corporate industry actors have undertaken their own research studies designed to provide favorable results. Investigations have repeatedly found manipulation by the tobacco industry, for example, to produce research findings showing smoking was less harmful than actually the case (c.f.: Wertz et al. 2011; Bero 2005; Cummings et al. 2007; Tong and Glantz 2007). Goldacre has similarly argued that the pharmaceutical industry is biased in its generation of evidence on product effectiveness, claiming “[d]rugs are tested by the people who manufacture them, in poorly designed trials, on hopelessly small numbers of weird, unrepresentative patients, and analyzed using techniques which are flawed by design, in such a way that they exaggerate the benefits of treatments…” (Goldacre 2014, p. xi). One study by Fries and Krishnan (2004) analyzed randomized trials of rheumatology drug trials, and concluded that ‘design bias’ led to an incredibly high proportion of trials (100 % of trails reviewed in their sample) showing positive results when that study had industry funding. Of course it is not only corporations pursuing monetary profit create evidence in biased ways. The growth in so called ‘creation-research’ (research specifically conducted to illustrate a divinely-created ‘young earth’) provides an example of how a particular religious belief has led to a body of research described as fundamentally flawed (technically biased) due to its starting premise to prove biblical literalism (Pigliucci 2002).

Technical bias in the selection of evidence

Technical bias can similarly occur in the selection of evidence, evidence is cherry picked so as to only highlight the pieces of evidence which support a desired outcome. Schön and Rein’s quote above noted the ability to ‘focus on different facts’, and there are numerous examples of where data can be strategically selected in policy debates. The supposedly ‘super wicked’ example of climate change, for instance, shows no shortage of accusations of strategic selection of data, no doubt contributing to intractability at times. Indeed, accusations of one specific form of cherry picking—strategic use of start and end years to show trends—have been made by climate change acceptors against skeptics (Institute of Climate Studies 2013), by skeptics against acceptors (Whitehouse 2014), or even by climate change acceptors against other acceptors as well [c.f. criticisms of the Obama administration’s use of data to show success at reducing emissions (Kessler 2014; Klare 2014)].

Technical bias in the interpretation of evidence

Finally, Schön and Rein’s quote also noted that facts can be interpreted in different ways—yet it is clear that interpretations of evidence can be technically biased where invalid conclusions are drawn from a particular body of (presumably valid or robust) evidence. There appears to be continual need for reminders that ‘correlation does not equate to causality’, or that an ‘absence of evidence does not mean evidence of absence’, such that these biased interpretations are so common. Misunderstanding risk statistics can provide another common case for biased interpretations—such as in the confusion of relative risk and absolute risk (Baron 1997; Barratt et al. 2004); e.g. a reduction in risk of 10 % does not equate to a 90 % chance of something occurring, yet it might be interpreted as such. One example of this was recently illustrated by Parkhurst et al. (2015) in a study looking at the use of evidence within the political debate over male circumcision for HIV prevention in Malawi. While clinical trials had measured a reduced (relative) risk of HIV infection in circumcised men of around 60 % (Mills et al. 2008), a local politician rejected this on the (inaccurate) grounds that there was still a 40 % (absolute) chance of infection in circumcised men. This interpretation appeared to support a political position, but provides a case of technical bias as the conclusion about absolute risk was simply invalid.

Issue bias

In contrast to these forms of technical bias, which are principally concerned with internal validity issues, issue bias instead reflects cases where the choice of evidence can bias a policy decision. This distinction is particularly important to address those who see evidence as playing a value-neutral role, or who argue that particular forms of evidence (such as RCTs) should be prioritized in political debates or on policy agendas—a logic that conflates intervention certainty with outcome desirability. As such, even when technically valid, issue bias can still occur in both in the creation and selection of bodies of evidence.

Issue bias in the creation of evidence

In terms of the evidence creation, issue bias arises in the decision of which outcomes to measure or evaluate. For years in the field of health, for instance, sociologists and policy analysts have identified how the choice of social categories or concepts to include in research (such as race, class, socio-economic status, etc.) will have implications for whether or how these issues are addressed in public health policies (Schwartz and Kart 1978; Krieger 1992; Navarro 2009). In addition, individual evaluation studies of social programs will also need to actively choose which social outcomes are measured in order to judge success—a choice that can shape the potential policy implications of the evaluation. A choice of relevant outcomes will always need to be done by some agent, yet the ways this choice can bias policy implications can be particularly obscured if evidence is assumed to be value neutral, or only judged based on its methodological rigor.

So, for example, gun control has been a persistently intractable problem in the United States for decades (Bruce-Briggs 1976; Kleck 1986), yet an appeal to evidence to provide a solution can be fraught with difficulty due to the multiple possible ways to evaluate the effects of gun laws. As Kleck (1986) has explained, research can look at a number of possible outcomes, such as assault-instigating effects, crime-facilitating effects, or assault-intensifying effects. Policy evaluations can also choose to evaluate either the effects of restrictions placed on the general public, or on specific ‘high risk’ individuals. There are a host of other outcomes of wider gun ownership that might also be included in an evaluation—from accidental deaths, to children’s tendencies towards violence, to citizens’ use of guns to resist violent crime or to privately apprehend felons—the choices between which will define the supposed ‘impact’ of any gun regulations. If one chooses to evaluate how criminals behave in relation to armed citizens (c.f.: Southwick 2000; Kleck 1986), this can lead to policy conclusions supporting gun liberalization, while evaluating death risks linked to widespread availability of guns (c.f.: Leenaars and Lester 1997; Burger 2002) typically leads to conclusions supporting greater control. As such the choice of what to measure can bias evaluation findings towards one or another political position.

Issue bias in the selection of evidence

A second form of issue bias involves the choices of which bodies of evidence are utilized to inform a policy debate or decision. While EBP advocates believe evidence can serve a rationalizing role, when there are multiple impacts of a policy, and multiple social goals at stake (as is typical for many wicked problems), claims that a policy solution is ‘evidence based’ can be equally made by opposing sides simply by selecting evidence of different social concerns. So for example, with regard to an infrastructure project there may be an evidence base about economic growth from similar projects, as well as evidence base about environmental harms. Supporters or opponents can each point to a body of evidence on which to justify their position. Again, this is not inherently a problem when considering that political debate rests on value differences, but it can lead to intractability in cases where opposed sides continue to appeal to ‘evidence based policy’ in a failure to recognize the underlying value differences.

A particularly relevant form of issue bias therefore can be seen in the contemporary EBP movement’s demands for policies to be guided by RCTs or by the application of so-called hierarchies of evidence (which place RCTs, or meta-analyses of multiple trials, at the top). It has been noted elsewhere that hierarchies of evidence were designed to judge methods used to demonstrate intervention effect, yet policy decisions typically involve a range of factors, the effectiveness of an intervention merely being one consideration (Petticrew and Roberts 2003). The prioritization of particular research methods (e.g. RCTs)—can therefore skew political attention to those issues conducive to experimental evaluation, or those strategies for which evidence has already been created (Barnes and Parkhurst 2014).

In health care, for instance, drug treatments are almost always evaluated through experimental designs. Yet there is an important question of whether treatment—and treatment through drugs—is necessarily the most important policy concern simply because it has the most robust evidence of intervention effect. Indeed, by 2001, the drug Sildenafil (marketed as Viagra) had been subject to no less than 16 randomized trials, every single one of which showed effectiveness over placebo for treating erectile dysfunction (Burls et al. 2001)—yet this overwhelming evidence of intervention effect clearly does not imply that provision of Viagra should be a health policy priority.

Instead, formalizing rules and norms that prioritize RCT evidence or impose hierarchies to guide political decisions can be seen to reflect Bachrach and Baratz’s (1970) idea of political power imposed through the mobilization of bias, described as: “[a] set of predominant values, beliefs, rituals, an institutional procedures (‘rules of the game’) that operate systematically and consistently to the benefit of certain persons and groups at the expense of others.”(p. 43). This is why a distinction between technical bias and issue bias is particularly important. While political manipulation or cherry picking of evidence can indeed be a problem from a technical perspective (one of internal validity), removing political considerations from policy decisions represents a form of depoliticization that many find undemocratic (c.f.: Young 2011). Indeed, as Wesselink et al. (2014) has explained, “[o]vert deference to EBP does not remove the need for political reasoning; rather politics is introduced ‘through the back door’ through debates on what is valid evidence rather than on what values should prevail” (p. 341).

Bias and intractability

Both technical and issue bias can contribute to issue intractability by undermining the ability for evidence to serve as a source of agreement and policy rationalization. Creation of technically biased pieces of evidence give competing advocacy groups different pieces of conflicting evidence to justify their views or positions, undermining scientific consensus or increasing uncertainty. Biased selection and interpretation of evidence can similarly perpetuate disagreements over what the evidence actually says or means. Intractability can also arise from issue bias, however, whereby value differences between competing policy actors lead to creation or selection of evidence related to one set of goals to the exclusion of others, with appeals to ‘evidence based policy’ serving to obscure those essential value differences. This permits multiple actors to take ‘evidence based’ positions without policy resolution. The increasing embrace of policies following hierarchies of evidence or RCTs further risks obscuring the political origins of contestation, leading to frustration amongst policy actors who may not realize their use of, otherwise technically valid, evidence has political implications from a perspective of issue bias.

Given how such biases leads to intractability, addressing these forms of intractability will not, necessarily, come from increased knowledge transfer or greater calls for evidence use. Rather, this paper argues that a deeper understanding of the political origins of these forms of bias is required to understand how their mechanisms of effect and where their impact may be mitigated.

Political origins of bias

Interests and contestation: overt politics of evidence

It has been argued that current calls for EBP fail to recognize that policy making deals with competing interests and multiple social goals. There appears to be an underlying assumption in much of the EBP movement that fidelity to evidence and scientific honesty will be valued in their own right—seen, for instance, in the continued surprise when evidence is misused, as well as the enormous efforts placed on efforts to ‘translate’ or ‘transfer’ research into policy—typically efforts that do not question whether anyone would not wish to use evidence, and which assumes that non-use of evidence must be because the evidence is either not known, or not understood (Rutter et al. 2013; Oliver et al. 2014; Caplan 1979).

Yet the understanding of policy making as dealing with the choice between contested sets of social values and goals has been a conceptual starting point of the policy sciences for over half a century. Easton 1971 (1953), for example, famously described political science as the study of the “authoritative allocation of values for a society” (p. 129). Similarly Brecht (1959) argued that politics is fundamentally about debates over what a good society looks like, a question that science cannot answer. Conceptualizing political debates as a process of competition to achieve policy results or achieve a desired societal outcome, therefore, leads to a very different perspective on how evidence can be expected to be used. There is no reason to assume that any competing actor or coalition will weaken their position for the sake of scientific accuracy. Indeed, when policy decisions can determine the political or financial survival of involved actors, we would expect groups to sacrifice scientific accuracy if doing so provides a competitive advantage.

From this perspective, many forms of evidentiary bias become increasingly predictable. The biased creation of evidence by industry actors noted above is a case in point. Corporations compete in the private sector to pursue profit. As such, we should expect private corporations to produce biased evidence in their interests if it might increase their competitive advantage. This is because any corporate actor who does not do so risks go out of business, replaced by an otherwise equal competitor willing to manipulate evidence to compete. The tobacco industry can again be looked to for a wealth of examples of evidence strategies aimed at achieving a desired policy outcome. One recent example is in how the industry has argued against policies requiring so called ‘standardised packaging’ (or ‘plain packaging’). Often the opposition to such policies has been highly visible—such as when Japan Tobacco International (JTI) placed full-page advertisements in British newspapers in 2013 (when the UK was considering this policy), claiming that the proposal was not ‘evidence-based policy’ but rather ‘policy-based evidence.’ [these advertisements were subsequently banned by advertising regulators for making false claims (Thomas 2013)].² Yet according to one analysis of the industry’s tactics in this policy debate, tobacco companies pursued an evidence strategy that insisted on unrealistic methodological perfection of evaluations conducted, that misquoted published studies, and that withheld evidence counter to their policy goals (Ulucanlar et al. 2014).

This inherent pressure to bias caused by financial interest is, of course, why conflict of interest statements matter, and why most scientific journals (and, indeed, many political bodies) require them. There can even be situations, however, where entrenched interests are so often challenged by research findings that a ‘rational’ political strategy can be to undermine faith in the whole of scientific practice in the first place—with Oreskes and Conway (2011) exploring how corporations have constructed doubt or undermined the credibility of the research process itself in areas such as tobacco harms or environmental issues.

It is not only corporate actors that may be involved in such strategies, however. Political competition faced by those in positions of authority will similarly provide pressure to use data in biased ways due to the need for a competitive advantage to gain, or remain in, power. In a pluralistic society, groups will compete for public support, and in individual elections, politicians will equally compete for survival through convincing enough of the public to vote for them. Unless scientific accuracy is a particularly important social value required to obtain citizen support (i.e. demanded by the populace), the nature of political competition will lead to scientific rigor taking a back seat to the more important goal of winning or surviving in political debates—which can manifest in technical bias. Such realities can help to explain examples of hiding or obscuring unwanted evidence by those in political power. In 2015, for instance, accusations of such practices were leveled against a number of national governments including: the Indian government for failing to release health survey data results that showed poor results in the Prime Minister’s home state of Gujerat (BBC News 2015); the UK government for failing to release numbers of people who have died within six weeks of having their government benefits stopped (Brown 2015); and the Chinese government for failing to report urban pollution data (Chen 2015). Each of these cases can be seen as a case of technical bias, as it represents a violation of scientific best practices of making all data available in a systematic and rigorous manner, yet presumably the risk of being accused of obfuscation was judged to be less politically harmful than the implications of making the data widely available.

Even in cases where technical accuracy is highly valued, and technical bias would undermine a political position, however, issue bias can still result from competition and contestation, as groups will focus on the relevant evidence (often in technically valid ways) that frames debates around their issues of interest. The competition between interests and ideologies therefore provide the incentives to undertake, accept, or maintain many forms of evidentiary bias detailed in the previous section. Indeed, these political origins of bias can be said to represent an overt politics of evidence as they arise from often deliberate strategies between groups that are openly competing in political spaces.

Cognitive shortcuts and motivated reasoning: subtle politics of evidence

In addition to bias arising from overt political competition between groups, there can also be biased uses of evidence that have less obvious origins. A more subtle politics of evidence can arise in particular when individuals are unaware how their value systems, or their group identities, bias their understandings and interpretations of evidence. Researchers in the field of cognitive psychology have particularly explored how heuristics (simplifying processes) can lead to biases in understanding information—typically defined as departures from ‘rational’ or purely analytical decision processes (often seen as incorrect conclusions from what evidence actually shows, or violations of accurate understandings of probability) (Gilovich and Griffin 2002). Such biases are often seen to manifest from decisions made under uncertainty, or when time and other pressures lead to intuitive judgments instead of more deliberate considerations of evidence (Gilovich and Griffin 2002; Kahneman 2011; Kahneman and Tversky 1974).

Some of the most common and important heuristics affecting affect how scientific evidence include the availability heuristic (drawing conclusions aided by utilizing the memory of similar cases) (Gilovich et al. 2002; Tversky and Kahneman 1973), affect heuristic (judgements influenced by existing affective feelings)(Finucane et al. 2000), and representativeness heuristic (conclusions derived from perception of similarity between a given situation and a prototypical one) (Gilovich and Griffin 2002; Kahneman and Tversky 1974). Common biases resulting from such processes include: Confirmation bias—Defined as: “the seeking or interpreting of evidence in ways that are partial to existing beliefs, expectations, or a hypothesis in hand”(Nickerson 1998 p. 175); illusory correlation (incorrect assumption of correlation) (Sternberg 1996); and attribute substitution, explained by Kahneman and Frederick (2002) as: “when confronted with a difficult question people often answer an easier one instead, usually without being aware of the substitution.”(p. 53). Finally, the concept of motivated reasoning has further been developed to explore cases of self-serving biases in which information, evidence, or arguments are understood in ways conducive to an individual’s desires, needs, or goals (Kunda 1990; Hahn and Harris 2014; Kahan 2011; Richey 2012).

There has been only limited application of these insights to the understanding of evidence in political settings, however, with much of the research in the field consisting of experiments in political psychology conducted in the US context. In one such study, for example, preference for a preferred political candidate was evaluated after providing negative information about the individual. Notably, Redlawsk et al. (2010) found that “voters may become even more positive about a candidate they like after learning something negative about that candidate.”(p. 564—emphasis in original).

Motivated reasoning research has also looked at how existing party affiliation affects interpretation of factual data about polarized issues (c.f.: Taber and Lodge 2006; Kahan et al. 2013), or perceptions of risk and of expertise—e.g. over climate change (Kahan 2014). Such research has even shown cases where greater scientific knowledge or numeracy could be correlated with increased bias in the interpretation of data. Kahan et al. (2013) for example, compared the incidence of invalid conclusions drawn from a set of data when the same information was presented as representing either the effect of a skin rash treatment, or the effect of a gun control policy. While the authors expected to see bias increase when data were framed to be about gun control, they further found that more numerate individuals (who tested higher on mathematical skills) were more likely interpret gun control data in biased ways.

Finally, Kahan (2013) also has reported on an experiment illustrating that individuals with greater cognitive reflection (measured in terms of ability to correctly answer mathematical questions which have an intuitive incorrect answer) had a greater propensity for motivated reasoning in an experimental setting. The author concludes that motivated reasoning may not necessarily be due to an over-reliance on heuristic (or intuitive) reasoning in all cases. In this case, he hypothesizes that “ideologically motivated cognition [is] a form of information processing that promotes individuals’ interest in forming and maintaining beliefs that signify their loyalty to important affinity groups”(p. 407).

These concepts help to understand the more subtle politics of evidence that explains additional elements of technical and issue bias beyond those which arise from explicit policy contestation. Individuals and groups with contrasting belief systems interpret and use evidence differently—not just because they are actively pursuing particular policy results, but because of how existing beliefs, motivations, and values structure the cognitive processes through which evidence is understood and applied. These theories can further help explain the source of the issue bias related to the prioritization of hierarchies of evidence or RCTs, which appears to be a form of attribute substitution—substituting the difficult questions of what to do to make society better with more straightforward questions of what interventions produce an effect.

While most of the political psychological research conducted on motivated reasoning has been done on populations of voters in the United States, we can also see the influence of cognitive in international expert bodies as well. The World Bank, for instance, dedicated a section of its 2015 world development report to exploring how its own expert staff were subject to cognitive biases which could lead to erroneous interpretations of policy-relevant evidence. These biases at times derived from existing ideological views of staff members; for instance finding that that pre-existing views on the importance of income inequality led to errors in the interpretation of hypothetical minimum wage data. Ultimately the World Bank concluded that its staff were subject to a range of biases that could undermine development planning, including confirmation bias, sunk cost, bias, and inaccurate predictions of the beliefs of the poor people they are meant to be serving with their work (World Bank 2015).

Intractability and evidentiary bias: a framework

The above sections have mapped out two distinct forms of bias—technical bias and issue bias—and discussed some of their political origins—deriving from differences in values, beliefs and ideologies, manifest through differing mechanisms, both overt and subtle. While some past works have attempted to identify elements of policy problems that might increase their ‘wickedness’—for example Head’s (2008), proposed components of complexity, uncertainty, and divergence—here I alternatively focus on elements already common to many problems (both wicked and non) which particularly can be seen to manifest in the forms of evidentiary bias described. These features, and the biases they engender, will in turn influence the intractability of policy debates, and the inability for appeals to evidence, alone, to resolve them. Three features in particular are discussed: policy complexity, contestation and polarization.

Complexity

One of the most common terms used in discussion of wicked problems is that of complexity. The term has been taken to mean many things, but a useful distinction comes from the field of complexity theory which often distinguishes between what are considered complicated as opposed to complex problems (Snyder 2013; Snowden and Boone 2007). In complicated problems there are many interacting or mutually interdependent elements, but each one may be solvable or knowable. A common example presented is sending a rocket to the moon (c.f. Glouberman and Zimmerman 2002): rocket science is difficult for many to understand, and reliant on many interacting pieces of expert information, but essentially predictable in most cases. Complex issues, on the other hand, are defined as having elements of uncertainty in causal relationships (many of which can only be identified after the fact) that prohibit prediction of similar outcomes if the same actions are taken in subsequent iterations. Raising a child is often noted as an example, as would be many experiences of social change (Kurtz and Snowden 2003; Glouberman and Zimmerman 2002).

In this conceptualization, complicated processes are typified by difficulty, while complex problems are typified by uncertainty. For many scholars, complicated problems are not necessarily wicked, as they can be ultimately solvable. Yet features of complicated problems can manifest in evidentiary bias, often in unconscious ways. Understanding the origins of bias for both complicated and for complex problems is, therefore, essential to address problem intractability. The distinction is particularly important given the origins of bias discussed above and the use of these academic fields to respond to bias. Cognitive psychology has identified different heuristics or mechanisms humans utilize in each of these cases. Kahneman (and colleagues) have famously described two forms of thinking—thinking ‘fast’ (intuitively) and ‘slow’ (deliberately and analytically)—noting the human propensity to think fast when tasks are complicated or difficult. Intuitive thinking can lead to biases such as attribute substitution and representative bias, even when, in theory, solutions are knowable (Kahneman 2011; Kahneman and Frederick 2002; Kahneman and Tversky 1974). When faced with uncertainty, thinking ‘fast’ can exacerbate different problems such as affective bias (Finucane et al. 2000), yet thinking ‘slow’ does not eliminate all uncertainty, meaning that other heuristics that deal with situations of partial information can still manifest in bias (Kahneman and Tversky 1974).

Complicated public policy problems furthermore typically involve multiple social outcomes of importance. Regardless of uncertainty in these cases, an increased number of relevant social concerns will increase the possibilities of issue bias, as there will be a greater number of evidence bases to draw on for each relevant concern. Complicated problems can also increase technical bias as multiple affective values will exist (with affective feelings towards each of the relevant concerns potentially leading to bias); and there may more opportunities for cognitive dissonance when values do not align between the multiple social outcomes at stake.

Contestation

A second source of bias arising from features of policy problems can be the level of contestation of an issue—in terms of how important that issue is to the groups in competition. As noted, fidelity to science (or scientific honesty) can be seen as one of the many social values that stakeholders in political arenas may possess. For policy debates with low levels of contestation (involving differences in opinions on values that are of marginal importance to interested parties), it would be unlikely that either side would sacrifice scientific credibility or violate the value for data accuracy to gain a political advantage. One would hardly expect groups to start manipulating research or cherry-picking evidence to influence policy decisions over which color to paint a new school, or to change the weekday that municipalities collect household waste, for instance. Yet when a policy debate revolves around an issue held to be of high importance, there is much a greater incentive for strategic technical bias. Corporate actors may see their company’s survival (or perhaps their profit line) as their most important interest. Hence a willingness to sacrifice scientific accuracy to produce evidence more likely to achieve desired policy outcomes. ‘Creation science’ has similarly no doubt grown from a recognition that mainstream scientific teaching poses an existential threat to the core religious beliefs of ‘young earth’ divine creation.

The level of importance to the individual will also influence cognitive processes, with issue importance part of what has been termed ‘attitude strength’ (Pomerantz et al. 1995). Studies of the affect heuristic, for instance, have shown that stronger attitude strength can increase biased information processing (c.f: Alhakami and Slovic 1994; Brannon et al. 2007). Cognitive dissonance theory similarly has illustrated a range of mechanisms used to avoid, dismiss, or ignore cases where values are inconsistent, and stronger value violations would be anticipated when evidence is brought to bear on highly contested debates. Indeed, Taber and Lodge (2006) found that stronger prior held beliefs on controversial subjects like gun control and affirmative action (in the US) led to increased biased reasoning, including reliance on strategies such as confirmation bias to avoid cognitive dissonance.

Polarization

The third and final component of the framework presented here is that of policy polarization. This can refer to how many viable options or positions there are for individuals (or policy makers) to take on an issue, but it can also reflect the polarization of the political landscape more broadly. In terms of polarized issues, at one extreme would be policy debates where there are a large number of intermediary positions. At the other would be policy debates where there appear to only be two diametrically opposed positions with no possibility for compromise. An example of a less-polarized issue might be market regulation. Extreme views no doubt exist, but the vast majority of people would fall along a spectrum of middle-range positions about which elements of markets should be regulated for public safety, or to address market failures. An example of a highly polarized issue might be that of abortion—which is often presented as morally wrong and unacceptable by its nature, or alternatively not a moral violation and as such a personal right for a mother. Other examples might be debates over the validity of including ‘intelligent design’ in a school science curriculum, or whether gay marriage should be permitted (or not). At times we see compromises made by politicians on these issues (offering ‘civil partnerships’ or abortion in particular circumstances), but in general the issues still are debated over a single binary division.

Policy environments may also be polarized as well, however. Kahan (2013), for instance, blames, in part, the polarization of American politics for what he sees as “ubiquity and ferocity of ideological conflicts over facts that turn on empirical evidence” (p. 407). Sunstein (2002) has explored group polarization and argued that individuals tend to move to more extreme positions when they deliberate with others sharing their own tendencies. This is said to come from “people’s desire to maintain their reputation and their self-conception” (p. 176). Kahan’s experiments in the American political context illustrate just how these pressures can lead to a biasing tendency in the interpretation of factual data, through what he terms ‘identity-protective cognition’—explaining “the goal of protecting one’s identity or standing in an affinity group that shares fundamental values can generate motivated cognition relating to policy-relevant facts” (p. 408). In highly polarized environments, then, the incentive for such identity-protective cognition to maintain group acceptance would be exacerbated. If we imagine individuals sharing political values, ideologies, and social interactions with a particular group, in a polarized society, an individual may risk losing their entire network and sense of community if they accept a valid (technically unbiased) evidence assessment which contradicts their group’s positions. The individual may have few shared values, ideologies, or social interactions with those on the other polarized position, and have a great deal to lose from group exclusion. In a non-polarized context, however, there may be several positions to adhere to, and multiple alternative groups in which to integrate.

Finally, polarization of the overall political landscape may also drive issue bias through the clustering of concerns that tend to split between divided groups. If valued social outcomes or concerns are highly clustered between opposed groups, we might expect that selection of issues over which to review evidence to more likely to come from within a single cluster, due to shared information and common thinking (and biases) within a the group.

Discussion

The contemporary evidence-based policy movement often sees evidence as a solution to public policy debates and a source of policy guidance. This fits a rationalist ideal of policy making that has been pursued for centuries, but which may fail to address many of the realities of policy making. Many features of wicked policy problems raise challenges as to how, or even why, evidence might serve a rationalizing role in such cases. Head and Alford (2008) have stated that “there has been very little attention to the implications for public management of how ‘wicked problems’ are identified, understood and managed” (p. 2—emphasis in original). This paper aims to help address this gap in part by exploring one of the key aspects of evidence use that may drive problem wickedness; those cases where seeming intractability arises from the failures of appeals to evidence to achieve solutions or resolve debates.

The paper has specifically delineated two forms of evidentiary bias that can undermine its use in these ways, thereby supporting intractability. It explored the political origins of those biases, discussing three particular features of policy problems that may manifest in bias through different mechanisms. Table 1 presents a summary of these findings in what can be termed a ‘cognitive-political framework’, due to its origins in both political studies and cognitive psychology³:

Table 1 Cognitive-political framework linking features of policy problems with sources of evidentiary bias.

There have already been some initial suggestions of ways to address policy wickedness in the literature. Head (2008), for example, has explained that “the three most widely recommended approaches to wicked problems—better knowledge, better consultation, and better use of third-party partners—deserve closer attention in future research”(p. 114). There have also been propositions for how particular approaches to deliberation can serve to address wicked problems. Carcasson and Sprain (2016), for instance, describe a process of ‘deliberative inquiry’ as one which is continually learning and facilitating interactions between rival parties; so as to provide information and consider multiple perspectives in order to reduce tensions between rival groups.⁴

The cognitive-political framework presented here presents an additional tool to consider when, and in which situations, different approaches such as these might help to address situations of intractability. For example, better knowledge may reduce bias arising from complexity, but may actually exacerbate bias in some cases of polarized motivated reasoning; and may similarly fail to resolve contested situations where issue bias allowing both sides to claim an evidence-based position. The deliberative approaches promoted by Carcasson and Sprain and others might also work to address some forms of bias explored in this paper—such as reducing issue bias by ensuring more explicit statement of values is made, or overcoming intuitive technical errors if time is taken to ‘think slow’ about issues—but other sources of bias may require additional intervention. E.g. cases of attribute substitution may need to be specifically flagged up if members of deliberative processes are unconsciously substituting easier to understand or answer questions for otherwise more relevant difficult ones.

Having a more explicit understanding of the origins and mechanisms of bias provided by the cognitive political framework may therefore inform the planning and strategies taken within efforts such as those deliberative problem-solving efforts aiming to overcome wickedness more generally. So if a problem is recognized to be complicated, for example, safety mechanisms, of a sort, can be emphasized to ensure that slower thinking (more conscious deliberation) is mandated where it is likely to be needed (Kahneman 2011). Issue bias derived from having multiple relevant social values might be addressed by inclusive deliberative strategies noted above, but they may further be managed by requiring relevant social outcomes to be explicitly named or agreed at the start of policy debates—akin to the processes of multi-criteria decision analysis (Belton and Stewart 2002), or indeed, the ‘mediated dialogue’ (B. W. Head 2008) or ‘frame reflective’ analysis (Schön and Rein 1994) that have been suggested to make the roles of values in policy debates more clear. Other sources of bias may be less amenable to single approaches, however. Taking the earlier example of contestation over what to measure in gun control evaluations, while it might be possible to include more of what different groups are concerned with (crime rates, violent tendencies, accidental deaths, etc.), there still may be motivated reasoning that rejects findings or leads to invalid interpretation of evidence, even by those with high levels of scientific knowledge. A combination of frame reflection, independent verification, depolarizing deliberation, and other approaches might be required to address intractability over evidence in such cases—as might indeed be expected for issues demonstrating multiple elements of wickedness, and multiple origins of evidentiary bias.

Much work remains to be done in this area, yet a more strategic approach to identifying the origins and mechanisms of evidentiary bias has clear implications for future areas of research. While this framework was informed by empirical examples, a next step could be more in-depth application and testing in specific case studies. Further conceptual work could also be useful, particularly as the scientific fields informing the framework continue to evolve or identify new mechanisms of bias arising from political arrangements. Another important piece of future research could be to systematically and critically review the state of knowledge about strategies that have been proposed or used to mitigate particular forms of bias covered by the framework. This could in turn lead to development and subsequent empirical evaluation of further strategies aimed at reducing intractability in particular political settings. Seeking to tackle the challenges posed by wicked problems will no doubt remain a priority in the policy sciences, however, and this paper hopes to provide one component that may be useful towards achieving that goal.