Introduction

In this chapter, we investigate the evidence perceived to be relevant to policy decisions for three contrasting health policy examples in Cambodia – tobacco control, HIV/AIDS and performance-based salary incentives. These cases allow us to examine the ways that policy relevant evidence may differ given the framing of the issue and the broader institutional context in which evidence is considered.

It is widely agreed, including within the global health community, that data and evidence are essential to inform policy formulation and implementation (Lavis et al. 2004; Katikireddi et al. 2011; Macdonald and Atkinson 2011; Franklin and Budenholzer 2009). However the rhetoric of evidence-based policy – one based on the assumption that research is objective or unbiased, and its uptake is a priori positive, with particular emphasis given to pieces of evidence classified at the top of so-called ‘hierarchies of evidence’ – has long been critiqued by social science scholars (c.f. Oliver et al. 2014a, b; Cairney 2015; Cairney et al. 2016; Wesselink et al. 2014; Guyatt et al. 2008; Tunis et al. 2003; Liverani et al. 2013; Smith 2013a; Smith and Joyce 2012; Smith 2014; Hawkins and Parkhurst 2016). For example, Weiss (1990) has argued that research alone ‘is almost never convincing or comprehensive enough to be the sole source of political advice’, and ‘there are always issues that research doesn’t cover’. Increasingly policy-studies scholars have explored aspects of the political system that may shape when, how and the types of evidence used within policymaking (Cairney 2015). These can include both how political institutions (such as formal structures, and less formal rules and norms) (Lowndes and Roberts 2013) and how key ideas (including the way that issues are framed and understood) influence which types of evidence appear to be relevant for, and are used within, different policy processes (Smith 2013a; Shiffman and Smith 2007; Parkhurst 2012).

However, as described by Oliver et al. (2014a), little empirical analysis has been undertaken of the processes or impact of evidence use in policy and the way that research and policy processes interact. This paper seeks to help address this gap, through a comparative examination of the role that institutional and ideational factors play in shaping evidence use for three contrasting health policy decisions within a single country context. Specifically this paper presents findings from research conducted in Cambodia, where the Ministry of Health (MOH), like many government departments in countries elsewhere (Cabinet Office 1999; Government Office for Science 2012; DEFRA 2011), has explicitly embraced the overarching language of using ‘evidence-based’ approaches to health policymaking. One example of this endorsement is in the country’s second Health Strategic Plan (2008–2015), which defines priorities and goals for the entire health sector, highlighting the need “to strengthen and invest in health information system and health research for evidence-based policymaking, planning, monitoring performance and evaluation” (Ministry of Health 2008). In this context, our study aimed to examine and compare the ways evidence was discussed or used in three contrasting health policy areas – tobacco control , HIV/AIDS and performance-based financing (PBF) – in particular for PBF we focus on a widely-praised government midwifery incentive scheme (GMIS) that was introduced to increase deliveries at public health facilities.

Tobacco control represents a policy decision for which there is a long history of acknowledged corporate and governmental financial interests that have often attempted to influence how health-related evidence is used in regulatory policy-making (Bero 2003, 2005; Ong and Glantz 2000; Tong and Glantz 2007). HIV/AIDS, on the other hand, is an issue with strong donor and global interest, and which has seen policy ideas particularly shaped by global civil society movements and consensus (Parkhurst 2012; Schneider 2002; Buse et al. 2008; Doyle and Patel 2008). Finally, PBF tends to have much less external contestation or debate, but is largely seen as a more technical matter related to health economics, health service provision or health systems functioning (Mills 2014; Meessen et al. 2011). As such, these three examples provide useful ways to reflect on how the different institutional settings in which policymaking takes place may influence evidence use, including how interests and ideas of key actors within the differing institutional arrangements play out in relation to evidence utilisation.

Methods

The paper draws on findings from in-depth semi-structured interviews conducted in Cambodia in 2015 and 2016 with stakeholders from key health sector organisations, as well as a related documentary analysis. The interviews were undertaken as part of a wider research project examining political aspects of evidence use for health policymaking in multiple countries. In case-study countries, key informants were first asked questions about the systems and processes through which evidence was used to inform health policy broadly, followed by asking for multiple examples of recent health policy decisions that could be illustrative of different aspects of evidence use. In all countries we subsequently investigated evidence use within tobacco control policy – given the importance of tobacco use for health in virtually every country context, as well as the existence of both a well-established evidence base and a global policy framework (i.e. the World Health Organization’s global framework convention on tobacco control). After consultation with local stakeholders, we then selected additional country-specific health policy decisions of interest or importance to enable comparative analysis. As noted earlier, this approach led to the selection of three examples in Cambodia: tobacco control, HIV/AIDS and performance-based financing.

Key participants were identified though purposive and snowball sampling strategies. In line with our approach, we first approached high-level policy makers likely to be knowledgeable about major policy developments across the entire health sector, and thus could provide a general overview of systems and structures in place to use evidence and advice on the selection of case studies. Subsequently, a scoping review of relevant documents (i.e. published studies and grey literature in the public domain such as policy documents and reports) was conducted to collect background information on each policy issue. This was followed by identification of individuals who could comment further on the use of evidence to inform the selected policy decision. We endeavoured to conduct interviews with people who represented a diverse range of perspectives for the health decisions investigated. In total, 26 participants were interviewed, including both government representatives as well as individuals representing influential stakeholders in the policy process, particularly from aid providers, non-governmental organisations, multi-lateral organisations, and local research institutes.

Interview guidelines focused on the following broad topics, which were tailored to the different roles of informants and the specific expertise or insights they would bring: (1) perceptions about the policy process, including the role of different actors and contextual factors; (2) the nature and source of evidence that was used to inform the policy decision; (3) the way in which evidence was presented and evaluated; (4) general views on institutional structures and practices of evidence use within the Cambodian health sector. Interviews were conducted face-to-face by the authors, recorded (if permission was given), and subsequently transcribed and coded into themes in an iterative process (Bourque 2004). Citations from interviews and documents are included in the presentation of results to illustrate key points and emerging themes.

Consent was obtained at the initiation of each interview, with respondents given options on levels of anonymity desired. Ethical approval to undertake the study was provided by the London School of Hygiene and Tropical Medicine; and research permission obtained from the Cambodia National Ethical Committee for Health Research (n. 0120; 06/05/2014).

Policy Studies Perspectives

It is now reasonably well-established that national policy contexts can vary considerably with important implications for evidence use. Yet even within a single country, the characteristics of evidence use for different health issues may also vary considerably. Previous work has made it clear that the political nature of policymaking means that there can be multiple competing interests and concerns at stake for any given policy decision – even within the health sector (Parkhurst 2017; Russell et al. 2008; Cairney 2015). This indicates that multiple pieces of evidence may be relevant or considered in the policy process, depending on the differing concerns at stake, rather than any single piece or body of evidence. Thus an important step in moving beyond an over-simplistic treatment of evidence use is to understand the differing interests of stakeholders holding varying power and influence over a given policy decision. Indeed, Cairney notes that there can be such contestation at each step of the policy process – from defining the problem, to deciding which evidence to generate (or presumably which evidence to consider), to choosing solutions (Cairney 2015).

Scholars have thus begun to apply a range of theories and concepts from the policy sciences to help deepen our understanding of evidence use given these realities. Pearce (2014), for instance, describes a ‘mistaken consensus’ that local climate policy can be based on emissions data, instead drawing out how ideas and arguments are also used, and needed to construct local policy responses. This view is similar to that of Wellstead et al. (2017), who argue that climate change adaptation science advocates are too narrowly functionalist in assuming that policies will change in response to feedback about climate change. Instead they argue that understanding policy changes in this area requires looking not just at the specific problems climate science identifies, “but also at the political and institutional factors that transform situations into problems and attempt to address them (p. 13)”.

This shift away from thinking about policy problems as fixed, but instead to consider how issues become ‘problematised’ directly draws on the field of interpretive (or critical) policy studies, which considers the roles of rhetoric or discursive framing in shaping policy outcomes (c.f. Fischer 2003; Bacchi 2009; Stone 2002). It is not just climate science, however, which has seen such developments in analysis. In looking at health policy, Smith (2013a), for example, argues that it is the roles and interplay of ideas (and ideas about evidence) that can be critical to understand evidence use within differing health-related concerns (Wesselink et al. 2014).

The policy sciences have thus been increasingly applied to questions of evidence use in health policymaking and beyond. These perspectives allow consideration of the multiple interests and multiple bodies of evidence that are important to a policy decision, while further recognising the ways that institutional and ideational factors can lead to differing constructions of what evidence is seen to be appropriate to address any given interest in the first place – with institutional forms and ideas closely linked to the relative influence of different stakeholders in policy processes.

In this paper, we embrace this approach, applying ideas from new institutionalism to explore the competing or contrasting constructions of evidence use for a set of three differing health policy concerns in the setting of Cambodia. On the one hand, new institutionalism highlights the not just the structures in place that shape decision making processes and outcomes, but also the importance of rules and norms within organisations that guide actor behaviours or decisions (Lowndes and Roberts 2013; Peters 2005). The approach also expands the focus of analysis beyond classic comparisons of state bureaucracies or legislative forms to consider the nature of institutionalised forces directing policy-relevant action across a much wider set of organisational forms, including non-state bodies, collections of stakeholders, or contrasting elements within a government system.

Applying such an approach to the question of evidence use for health thus allows us to focus on multiple issues. First we can consider the power or influence different stakeholders have over policy processes based on their structural positions for a given policy issue – reflecting on how different bodies of evidence may be more or less relevant to given stakeholders with influence. This approach also, however, allows exploration of the institutional logics which those stakeholders possess (c.f. March and Olsen 1989, 2006) that further shapes uses and understandings of policy-relevant evidence. In order to achieve these goals we first provide an overview of the three policy areas addressed, followed by a description of the types of evidence seen to be applied or important in each case. This is then followed by our analytical section that applies this institutional and ideational lens to explore such questions.

One Country, Three Health Policy Issues

Tobacco Control

Tobacco smoking became increasingly prevalent in Cambodia in the 1990s when the country was recovering from its civil war. At this time, there emerged the presence of many transnational tobacco companies in the country, the most prominent of which was British American Tobacco (BAT). The need for foreign investment and lack of regulation of advertising at this time, was explicitly recognised by British American Tobacco (BAT) who described Cambodia as “an attractive and strategically important target” (Mackenzie et al. 2004). A 1993 BAT industry plan, for example, acknowledged that awareness of the relationship between smoking and morbidity/mortality would increase in Cambodia through the activities of the World Health Organization (WHO), but estimated that “the significant revenues generated by tobacco advertising [for the government] will, in the short term, delay anti-smoking initiatives until alternative forms of revenue are guaranteed” (Mackenzie et al. 2004). BAT’s preferred option was reportedly to become a majority shareholder in a joint venture alongside local interests. Such an arrangement would presumably allow industry control of the composition of company board of directors and significant influence over corporation activities, whilst also encouraging a local stake in the corporation’s success. BAT achieved this in 1995 (Mackenzie et al. 2004).

According to Mackenzie et al. (2004) there was also at this time, owing to the lack of regulation, huge scope for tobacco-control advertising and promotional activities (Mackenzie et al. 2004). Indeed, a 1994 survey of twelve main streets in the country’s capital Phnom Penh recorded 49% of the advertising signs (8495 in total) were advertising tobacco products (Smith 1996).

HIV/AIDS

HIV/AIDS in Cambodia has a very different history to tobacco smoking. In the mid-1990s, Cambodia had one of the fastest growing HIV prevalence rates in Southeast Asia, with injecting drug use and commercial sex driving HIV transmission (Weiss and de Cock 2001). Adult prevalence peaked at approximately 2.0% in 1998 (Pean et al. 2005). Since then, a number of prevention and treatment programmes have been introduced, however, and the country’s prevalence has reduced, to an estimated 0.7% in 2013 (UNAIDS 2015; Vun et al. 2014).

The response has been divided into three phases: in phase I (1991–2000), a nationwide HIV prevention programme targeted brothel-based sex work, introduction of voluntary confidential counselling and testing and home-based care, and peer support groups of people living with HIV emerged; phase II (2001–2011) was characterized by expanding antiretroviral treatment (covering more than 80% of the population) and continuity of care, linking with other health services, accelerated prevention among key populations at higher risk (entertainment establishment-based sex workers, men who have sex with men (MSM), transgender persons, and people who inject drugs), engagement of health workers to deliver quality services, and strengthening health service delivery systems; and phase 3 (2012–2020) aims to attain zero new infections by 2020 through sharpening responses to high-risk population groups, maximizing access to community and facility-based testing and retention in prevention and care, and accelerating the transition from vertical approaches to linked/integrated approaches (Vun et al. 2014). In recognition of the country’s success in halting and reversing the spread of HIV (relating to the United Nations Millennium Development Goal or MDG 6), Cambodia was in 2010 presented with an MDG Award (UNAIDS 2010).

Performance-Based Financing, and the Case of the Government Midwifery Incentive Scheme

The final health issue we explored in relation to the use of evidence was that of performance-based financing (PBF), with specific discussion in interviews about the role of evidence in supporting the government midwifery incentive scheme (GMIS). In many low and middle-income countries, PBF is increasingly being used to redress particular aspects of health system underperformance, particularly the productivity and quality of healthcare providers. It involves offering incentives intended to redress underperformance, particularly high worker absenteeism, which is frequently observed in poorly funded public health systems with poor accountability (van de Poel et al. 2016). Support for PBF has spread rapidly in many countries in recent years (van de Poel et al. 2016). But whilst there is considerable enthusiasm for PBF policies, according to a Cochrane Collaboration review (2012) of pay-for-performance to improve the delivery of health interventions in low- and middle-income countries, the current evidence base is too weak to draw any general conclusions regarding effectiveness, with more robust and comprehensive study needed (Witter et al. 2012; van de Poel et al. 2016).

According to van de Poel et al. (2016), Cambodia was the first documented case of a low-income country to experiment with PBF of public health care. Since 1999, a variety of health programme funding of districts and facilities in Cambodia have been contingent on performance targets or have directly linked revenues to services delivered. The main PBF programmes implemented have specified performance targets relating to child vaccination, antenatal care, delivery in a public facility, and birth-spacing use. These funding arrangements have been intended to increase aspects of healthcare provision, and there has been considerable variation in the strength and conditions of the incentives offered (van de Poel et al. 2016).

The interviewees specifically identified the GMIS as a notable PBF policy, and described how the policy contributed to reducing Cambodia’s high maternal mortality ratio (MMR) over recent years. The GMIS became operational nationwide in late 2007, following a joint prakas (directive) from the MOH and MEF to allocate government budget to the incentive payments (Ir and Chheng 2012). The UNFPA was considered to be behind the policy change, for example through supporting a High-Level Midwifery Forum in late 2005 that bought together representatives from several government departments. However it was the prime minister who reportedly ‘gave the green light’ for the policy to go ahead. Other stakeholders were not thought to have had much direct influence over this decision, and the Cambodian Midwives Council, for example, was established after the implementation of the GMIS.

The GMIS aimed to boost facility deliveries by motivating skilled birth attendants (or trained health personnel) to promote deliveries in public health facilities. It did this by providing midwives (and other trained personnel) cash incentives based on the number of live births they attended in public health facilities – USD15 for a live birth in a health centre and USD10 for a live birth in a referral hospital. The reason for the higher payment in a health centre than a hospital was to provide a stronger incentive for deliveries at health centres – the recommended facility for normal deliveries to be managed (Ir et al. 2015). According to the MOH’s guidance, besides midwives, physicians and other trained health personnel can also receive these incentives when attending deliveries in public health facilities. Up to 30% of the incentives will be shared with other health personnel in the facility and eventually with other people such as traditional birth attendants (TBAs) who refer women to the facility for delivery (Ir and Chheng 2012). The number of deliveries is reported monthly by health facilities through the routine health information system. Based on the number of reported deliveries, incentives are disbursed quarterly to the facilities through public financial disbursement channels (Ir and Chheng 2012).

The Nature of Evidence Used

In this section we begin to describe and examine the reported differences in evidence use between the three policy areas. The evidence relating to each issue can be categorised in various ways, including by evidence topic (e.g., health, economic) or type (e.g. epidemiological, pilot study), which relate to the issue framing by key stakeholders, and the sources relied upon (e.g. global literature, national statistics, government survey). What is clear, however, is that no single uniform construction of policy relevant evidence was seen across cases.

Tobacco Control

Global evidence on tobacco harms were at this time considered well established, but local data on smoking rates were fairly limited. In the late 1990s, Cambodia had some small regional surveys of smoking prevalence, but it wasn’t until 2005 that accurate nationwide prevalence data on tobacco use were available (Singh et al. 2009). In spite of a lack of local data on smoking, in May 2004 Cambodia signed the Framework Convention on Tobacco Control (FCTC), a global policy agreement that calls for a number of restrictions on tobacco advertising and promotion – restrictions which many global health authors present as ‘evidence based’ (Myers 2013; Rosen et al. 2013; Glantz and Gonzalez 2012). Many stakeholders interviewed noted the importance of the FCTC locally, as it dictated that the government could not engage with industry on developing tobacco control policy. However implementation of the elements of the convention were described as only occurring slowly or in limited ways, which interviewees suggested was due to industry influence.

For example, one independent health sector consultant explained:

The tobacco industry and lobby is massively powerful here. (IDIFootnote 1-01, June 2014)

Another respondent, a senior civil servant in the MOH, explained:

We don’t know exactly why the law is very slow to be approved. Probably this is also due to lobbying of tobacco corporations, but we don’t have evidence to prove it. (IDI-10, August 2014)

This individual also noted that tobacco control did not appear to be a priority in the national health sector strategic plan.

The context of a deeply entrenched and powerful tobacco interest was also manifested in how respondents conceptualised the evidence that was relevant for moving tobacco control policies forward. A number of civil servants interviewed, for example, stressed the need to counter other evidence the tobacco industry uses to frame tobacco control in a way that suits industry interests. For example:

They [tobacco corporations] always complain that if we increase taxes, farmers will lose their job. So, you have to explain to the government that, if you increase taxes, the margin will not affect the industry. Also, we have to explain that farmers do not rely on one crop only, so reduced tobacco production will not significantly affect them. (IDI-06, June 2014)

Tobacco industry is powerful and has money. Some people are lobbied by tobacco corporations. They [tobacco corporations] have a lot of experience. They can approach friends or members of the family and get confidential information about policy making. Then, people that are lobbied create opposition at the inter-ministerial meetings. They often say that tobacco control will impact on the economy and farmers. (IDI-08, August 2014)

Respondents also emphasised the importance of making different evidence-based arguments to different actors. In particular, it was noted that the Ministry of Economy and Finance (MEF) needed different evidence regarding tobacco control to the MOH to try to convince it to support policy action. As one civil servant explained:

You have to find a way to convince people… also because policy is multisectoral. It’s not that one minister decides. If you want to increase tax, this is not an issue of the Ministry of Health. We don’t have the power to do this. We can do a smoke-free policy, but tax is under the Ministry of Finance. So you have to work closely with the Ministry of Finance. In Cambodia, when you talk to the Ministry of Finance, first you have you prove to them they can make more money… The industry can say ‘oh if you increase taxes, you will lose revenue’. And you have to present evidence that increasing taxes is not a loss of revenue. (IDI-06, June 2014)

We explain to the government that an increase in tax does not change the overall volume of cigarettes that are sold in the market. The case of Thailand shows this. Why? Because smoking prevalence decreases, but population increases. Cambodia is the same. Smoking prevalence has gone from 49.6% to 42.6%, however the absolute number of smoking is always 2 million because population increased… and you have to tell the government these facts. So you have to do a lot of work with the government to prove this. And of course, the tobacco industry makes a lot of money. We cannot stop them. (IDI-06, June 2014)

This civil servant also explained how evidence from neighbouring countries was considered influential, as was the normative element of the FCTC.

Usually in Cambodia we present evidence or examples from ASEANFootnote 2 countries. How is Vietnam doing? Thailand? Indonesia? Then, we also do international. But ASEAN is very important, also because we are approaching the ASEAN [Economic] Community in 2015, and member countries do not want to be left behind. (IDI-06, June 2014)

Overall, whilst many of the respondents spoke of the slow progress of tobacco control policy in Cambodia, the government has made substantial progress in tobacco control by banning the advertisement of tobacco products in 2011 as well as smoking in workplaces and public spaces in 2016 using sub-decrees. However it wasn’t until April 2015 that the Cambodian National Assembly passed the country’s first-ever law on tobacco control, which was ratified later the same month. The new law tackles tobacco from a variety of angles, including through import and sales restrictions, and bans on sales to minors and pregnant women (FCTC Implementation Database 2014).

HIV/AIDS

HIV/AIDS policy-making illustrates a radically different political context in which the utilisation of health policy-relevant evidence can be explored. The United Nations AIDS programme (UNAIDS) has stated that Cambodia has “used high-quality strategic information to inform a [successful] evidence-based response” (UNAIDS 2012); and interviews stood in dramatic contrast to those with stakeholders advocating for greater tobacco control who expressed the need to develop or discuss evidence of financial impact (e.g. on farmers or the treasury) to justify policy action. Instead, in discussions of HIV/AIDS, whilst a variety of evidence types were clearly brought to bear, NGO and donor-organisation respondents discussed how it has often been epidemiological modelling and cost-effectiveness analyses (IDI-21, 19, May 2016) – forms of evidence more typically advocated by public health actors for priority setting – that were seen as important evidence to guide policy. One of the respondents spoke about how this approach should be replicated in other areas of health policy-making:

I would say for HIV/AIDS it’s that way [evidence use more technical than in other areas of policy] every time. I feel like that’s brilliant and the model of HIV/AIDS [should] be replicated to other disease, for example we still have a very high number of death among pregnant women, the baby, the infant. So why don’t they learn from the HIV/AIDS program. (IDI-21, May 2016)

When interviewees were asked about the use of evidence within particular policy developments, a range of evidence types were described, in addition to epidemiological modelling and cost-effectiveness studies mentioned above. Other relevant evidence was said to include pilot studies, used for example to inform the development of a community-based testing approach implemented in 2013, where the HIV testing is performed by lay counsellors – volunteers from population groups at higher risk of HIV infection. National prevalence estimates and international evidence were also evidence types that were frequently mentioned, particularly with international evidence from other Southeast Asian countries, and particularly Thailand. The importance of international evidence may also reflect the strong role of donors in HIV/AIDS policy-making in Cambodia.

Yes they [policymakers] welcome [overseas evidence] in the HIV area. I don’t know about the other area. They welcome to learn the best practice from the region. This week one [NGO] staff member. He joined the field in Bangkok. (IDI-20, May 2016)

Epidemiological modelling of future prevalence scenarios has been considered key to informing policy as to the prioritization and targeting of interventions, preparation of operational plans, budgets and resource mobilization efforts (11) (12), and cost-effectiveness analyses such as modelling has highlighted areas where technical efficiency might be improved. The National Centre for HIV/AIDS, Dermatology and STI (NCHADS) within Cambodia’s MOH, which is responsible for the health sector response to HIV and other sexually transmitted diseases, has led the analysis, in collaboration with relevant departments and centres of the MOH, the National AIDS Authority and other government institutions, as well as health service providers, non-governmental and other civil society organizations, and development partners.

When asked about the reasons why HIV/AIDS policymaking stood out in terms of the use of what is more typically considered policy-relevant health evidence, respondents particularly spoke of the strong donor interest and support for HIV policy-making in Cambodia. They felt that this was key to driving the type of evidence being used in policy-making, and also the relatively well-functioning institutional entry points for such evidence, including here the relevant technical working groups (TWGs) of the MOH. One NGO respondent explained:

Yes it’s different [the policy-making process for HIV/AIDS compared to that for other health issues]. I think this is because donor support, and I think the other thing is because of resource, donor support and resource. Resource, I would say financial resource and human capacity resource, let’s say for HIV/AIDS they have more educated [staff] and they adhere to plan, they adhere to target and they target evidence and I feel like the government take that approach very well, participatory approach, it’s very well, because it’s an emergency situation but it is also in the situation where funding is allowing so that’s why we feel like they are open. (IDI-21, May 2016)

However there were downsides to a donor-driven approach also described, including in relation to siloed, non-integrated evidence gathering. When asked what could be improved, one interviewee explained:

Well I think it’s coordination. Because there’s basically the different donor programme, donors put all the evidence together. It sits on programmes that are… like some donors who are actually doing their own evidence, but not systematically led by the national programme and disseminated in a timely manner. (IDI-19, May 2016)

Interestingly, while HIV/AIDS policymaking has at times been seen as controversial or contested in some countries, related to the highly stigmatised nature of HIV transmission in some contexts (Rankin et al. 2005; Mahajan et al. 2008), we found little evidence of this in Cambodia. Whilst stigma and discrimination towards groups at higher risk of infection (sex workers, men who have sex with men (MSM), transgender persons and people who inject drugs) were noted, these were perceived to be relatively low compared to in many countries elsewhere. Instead, one of the NGO respondents explained that in Cambodia policy-makers are relatively open – and increasingly so – to discussing these groups and considering evidence relating to these groups.

The policy-makers they are more open now… for HIV the policy maker they are more open and learn from the [experience of high-risk groups]. (IDI-20, May 2016)

The respondents from a key NGO also spoke at length about the effort made by the NGO to engage high-risk population groups in the policy-making process, particularly in regard to supporting representatives to speak at community meetings and at the MOH TWG meetings.

[The NGO] work to promote that involvement in the policy-making as well, not only [the NGO] but also civil society. But we try to involve the key representatives from each key population to enrol in the policy-making process… we use the number, we use the finding, we use the civil society. But also bring the key population to talk during the meeting is more powerful… So this is like an MSM person or a sex worker, an entertainment worker could stand and speak about their challenges and the law enforcement people they listen to this. They’re part of the meeting. Everyone is part of the meeting… They also listen and sometimes they [describe] their challenges and you can see some improvement. (IDI-20, May 2016)

Performance-Based Financing, and the Case of the Government Midwifery Scheme

In contrast to the evidence types seen as relevant for tobacco control and HIV/AIDS policy-making, when asked about evidence for policy-making in regard to PBF schemes generally, these schemes were described as reliant almost solely on evidence from pilot studies. Interview respondents spoke of various pilot schemes of PBF that had been run over the years in different districts and by different groups, often by non-governmental organisations (NGOs), but also in workplaces of NCHADS. The perceived dominance of pilots as an evidence type is likely due to specific PBF policies being scaled up based on a pilot, but such schemes are likely also informed by evidence from health economics more broadly (and indeed, from basic microeconomics) that incentives can achieve outcomes (Mankiw and Taylor 2006). When pushed for further examples of evidence use in the PBF area, some respondents mentioned evidence from the Demographic and Health Survey, and also international evidence as informing the use of such schemes – but respondents didn’t provide specific examples of such evidence. Some mentioned the low payment of midwives and the lack of incentives for women to deliver in health facilities as evidence for needed change.

I don’t think anyone guided the government to design that policy. But it came clearly from many dialogues that the pay was not enough, that the arrangements did not encourage midwives to work in remote health centres, and did not encourage mothers to use health facilities. (IDI-11, June 2014)

A few years ago there was a policy to put one midwife in each health centre and the midwifery incentive… Hun Sen acted on this, and the policy was implemented immediately and very effectively… the. trigger was the Demographic and Health Survey… it has quite a bit of impact, and there was a lot of pressure from the international community… it was a relatively ‘easy fix’, a simple solution. (IDI-12, June 2014)

However, the dominance of pilots as an evidence type fits with observations from van de Poel et al. (2016) above, of Cambodia’s pioneering role in experimenting with PBF of public health care, and also with one of our respondent’s description of Cambodia as ‘a country of pilots’ (IDI-23, May 2016).

In contrast, however, there was also considerable discussion of that at times policy directives come from high levels of government – within the MOH, or as a decision made by the Prime Minister himself – and that in such situations evidence is perceived to be of limited importance. The GMIS was described as an example of this at times:

That [the GMIS] was an example of policy being changed by the government. It’s the government’s job, without any evidence. (IDI-15, May 2016)

You know, in the United States the evidence has, as far as can tell, no effect on congress. But what happens is they pass laws and then health and human services when they’re putting out the regulations or something, that’s where the evidence comes in. Here it’s more like everything at the congress level, even there is no, it doesn’t get more rational as it comes down through the MOH… People have very set ideas about things and those aren’t going to change no matter what evidence is put in front of them. (IDI-22, May 2016)

You can present the evidence and present it passionately and you can present it unanimously when you are heard, but there’s never really a proper policy dialogue. So saying well I could go and speak to the Ministry of Economy and Finance about that, we need extra money or we need to look at the budgets or perhaps we need to revise the [pre-service] training curriculum. You don’t get that. (IDI-15, May 2016)

We don’t even know who made it. There’s a re-writing of history that claims the MOH thought about it but there was no evidence. I was here right after it started and know many people here when it started. At the time no-one was claiming the MOH invented it, so it came out of the MEF, the Ministry of Economy and Finance. It was actually hugely successful. (IDI-22, May 2016)

Another respondent from a donor agency was unable to name evidence in support of the policy, and instead described how he saw the policy process for the GMIS.

I think in Cambodia evidence [is not so important, rather] government want the community to deliver their baby at the health facility so the government just simply providing incentive to the health staff, community wide, so for delivery of one live birth delivery, they get $15.00 this is the decision by the government and government budget and then see if they implemented that. (IDI-17, May 2016)

This perception of success appears to come from data showing increasing facility utilisation for delivery and falling mortality rates nationally after implementation of the programme. Since then, the percentage of deliveries in public health facilities has increased substantially, from 29% in 2006 to 57% in 2011, and the MMR has declined substantially from 473 per 100,000 live births in 2005 to 206 in 2010 (Ir and Chheng 2012). Care, of course, is needed with interpretation of such evidence. A number of evaluations of PBF schemes such as the GMIS have been undertaken, and PBF policies have been credited with developments including increasing utilisation by the poor, decreasing total family per capita health expenditure and encouraging better management (Eldridge and Palmer 2009) – but drawing firm conclusions of causality can be problematic, particularly when such programmes have been implemented alongside other health sector reforms (Soeters and Griffiths 2003). One respondent further commented that the quality of evaluations undertaken is often poor.

Institutional Features and Logics of Evidence Use

The three policy areas presented show few similarities in how pieces of evidence were used in various aspects of policy making, despite all being discussed or undertaken within a single MOH, and within a broad policy environment in which ‘evidence based policymaking’ is rhetorically championed. In this section, however, we draw out some of the particular institutional and ideational features of the three health policy concerns that may help to explain these findings.

A starting point is to compare the institutionalised positions of influence of the key stakeholders in each case, to reflect on how the relevance of particular evidence types fit with the interests of such stakeholders. This can then be followed by considering any contrasting institutional logics that similarly might help explain differences in evidence utilisation. Such logics could either be direct thinking about which evidence is relevant and why (such as how public health actors explicitly embrace hierarchies of evidence at times), or they may be related to the overarching goals or expectations of the actors involved, which subsequently shapes their uses of evidence (such as when particular types of evidence more naturally align with or fit broader goals).

In the case of tobacco control, the historical influence of the tobacco industry appears particularly relevant, and the nature of contestation for this issue appeared to principally be framed in terms of financial implications of tobacco control. Our respondents described the financial importance of tobacco for the agricultural sector and the national economy as the paramount concerns for any policy change. Tobacco control were well aware of the need to present different evidence and frame the issue differently to address the concerns of the most influential stakeholders – with a particular distinction made between the health and economic evidence needed when speaking to policy-makers from the MOH and MEF, respectively. The need for taxation and other regulatory policy to be made outside the MoH illustrated how limited health-related evidence of tobacco harms could be in driving tobacco control policy forward on its own.

Evidence from neighbouring countries, and the FCTC, were said to be influential. But even so, and despite considerable progress, policy change in line with these was described as particularly slow, as too was the development of what were considered more appropriate forms of evidence to guide tobacco policy from a public health perspective, such as national smoking prevalence surveys. Despite global evidence on tobacco harms and increasing local data on smoking, it was the concerns regarding economic growth and the industry’s entrenched interests and lobbying documented in our study and elsewhere (Mackenzie et al. 2004; Collin et al. 2004; Mackay 2004) that dominated the agenda. As such this significantly appeared to slow down the translation of the FCTC into local tobacco control policies; as well as the collection of, or action based on, forms of evidence typically seen as relevant to health promotion.

In contrast, the HIV/AIDS policy response in Cambodia developed in a rather different political context. With HIV/AIDS, there was no establishment of corporate interests, and little obvious financial interest at stake for any major stakeholders. Instead, the issue may have achieved a relatively high level of priority for policy action in Cambodia due to the attention and resources this issue has been accorded by donor agencies, and possibly related to this, the well-functioning TWGs of the MOH for HIV policy-making, as described by our respondents. The institutionalisation of donor influence is, in fact, reflected within the country’s various strategic documents for guiding programme implementation – including the National Strategic Plan for HIV/AIDS, 2011–2015 (NSP III) and Cambodia 3.0, a strategy developed by the country’s Ministry of Health to eliminate new HIV infections and congenital syphilis by 2020. Both are considered to be in line with the global targets and foci established by UNAIDS and the US PEPFAR programme (from which Cambodia is a recipient of funds) (PEPFAR 2015; UNAIDS 2015). These are highly technical global policy agencies, however, who routinely promote, or operate based around, particular forms of evidence types – embracing international discourses of evidence-based policy making. This may translate to the Cambodian context, particularly if there were no other strong interest groups to present alternative rhetoric or framing around the issue – and as such may have led to the use of evidence types in Cambodia more typically advocated by public health advocates in HIV/AIDS policy-making, and the observation of one NGO respondent that the National HIV/AIDS Centre is ‘big on evidence’ (IDI-25, May 2016).

Indeed, while in many countries the issue of HIV testing has been subject to debate or controversy – particularly around issues of disclosure or confidentiality of people living with HIV, and the challenges associated with addressing HIV in oft-stigmatised groups such as men who have sex with men, transgender people and sex workers – these concerns were considered by our respondents to have been relatively unimportant in Cambodia (even if admittedly seen as sensitive). That HIV/AIDS is an issue with social connotations in Cambodia perhaps explains the use of narrative evidence – stories of the lived experiences of marginalised groups – to influence policy-making. However, the relatively low level of moral contestation for this issue in Cambodia was noted – and used to explain the recent introduction of community-based rapid HIV testing (so-called ‘finger-prick testing’) by lay volunteer counsellors for high-risk population groups. Within this recent national policy, however, it was again evidence of effectiveness provided from a pilot study that could be seen to lead to policy change (KHANA 2014; Ministry of Health 2012).

The importance of pilots, however, was much more apparent, and described as the primary source of policy-relevant evidence for PBF. This was in contrast to tobacco policy appearing to require discussion of evidence of financial impact (linked to the influence of one set of interest groups), and HIV policy drawing particularly on epidemiological models and surveys (in line with norms and expectations of global health agencies). Again, however, we can look to the most influential stakeholders involved and their institutional logics to help explain the emphasis on pilots as a form of evidence in this case. Indeed, this helps to move away from the oft-criticised over-reliance on the idea that a single hierarchy of evidence can guide policy decisions, to instead consider the policy ‘appropriateness’ of particular forms of evidence (Parkhurst and Abeysinghe 2014; Dobrow et al. 2004; Young et al. 2002).

Unlike the previous cases, the GMIS policy appears to have had few stakeholders outside the government itself. It was reportedly made from the highest levels of the government, with some interviewees speculating that it was driven by the Prime Minister’s office in particular in response to a feeling that some action must be taken to help achieve the maternal health millennium development goal by 2015. The power and influence in this case appeared to be particularly hierarchical, with decision making made through a planning and management orientation. Pilot studies, which examine feasibility of an intervention, are a first step in exploring novel interventions, novel applications of an intervention, or the feasibility of an intervention in a particular context when the effectiveness of that intervention may be context dependent (Leon et al. 2011). For this reason, they are often considered important evidence of effectiveness as well as feasibility within a particular context and for complex interventions, on the premise that cultural appropriateness of interventions is important and can shape outcomes (Bernal et al. 2009). As such, pilot studies have been described previously as particularly applicable in health services and health systems research (Craig et al. 2013), but they also could be seen to be particularly relevant when a government has decided that wide-scale implementation of an intervention is a primarily objective at hand, such as achieving an MDG target through reducing the high maternal mortality rate.

In Table 2.1, we present a summary of these findings. In particular we highlight the stakeholders established to have dominant influence in each policy case, and their institutional interests and logics that help to explain the evidence said to be used by our interviewees.

Table 2.1 Characterisation of the institutional and ideational factors related to the health policy issue
Full size table

Discussion

Despite a common use of the language of evidence-based policy making in the health sector, there are in fact many types of evidence which can speak to a variety of political concerns and mandates that play out in the policy process. Sometimes evidence use differs because the evidence needs differ to solve an agreed upon problem. At other times, there may not be any agreement on the nature of the problem, however, and as such it is features of power, interests, and framing that serve as important drivers shaping which evidence informs policy considerations. An institutionalist approach, however, can help to understand some of these dynamics. It can identify which stakeholders have established positions of power within differing health policy issues, linking their interests to the nature of evidence used. It can also reflect on the institutional logics of these stakeholders which further may influence when or how evidence is utilised.

For tobacco, large and expensive national prevalence surveys were considered necessary evidence for intervention, even given considerable evidence from smaller studies of a high prevalence of smoking in the country, and the irrefutable global evidence linking tobacco to numerous diseases and mortality. Such surveys, however, were needed because of a demand for evidence that could speak to the dominant concerns of financial impact and the logic that evidence was needed to illustrate economic impact, rather than any public health logic of evidence to show medical harm to the population. The importance of the ministry of finance is thus apparent in this case – illustrating both its dominant policy concern in terms of the economy, but also its logic of what forms of evidence are required to speak to that concern. Furthermore, it is of course critical to understand the historical influence and role of the tobacco industry in the country, which no doubt has played an important role in shifting the terms of tobacco policy to one of revenue.

In contrast, for the case of HIV/AIDS, the dominance of global donors in supporting this health issue, and the apparent limited contestation at a local level, appears to have led to the explicit embrace of epidemiological evidence that is widely held to be appropriate for HIV planning within the global health community. In the final case of PBF, however, it was the government that drove both the initiation and implementation of the policy response. This state-controlled process appeared to reflect a belief that national action must be taken to address an existing priority (in the form of a Millennium Development Goal). This in turn naturally led to a logic which saw relatively micro studies focussed on implementation to be the most policy-relevant. Although it is worth noting that in the case of the GMIS, some believed that evidence was not perceived as important at all, due to the policy being driven by higher level political authorities. Indeed, some stakeholders simply referred to the GMIS policy when asked for good example of the use of evidence because it was national action based around the ‘evidence’ that maternal mortality rates were too high – a much simpler logic of evidence informed policymaking whereby evidence of a problem is seen to justify widespread action, in contrast to more traditional health sector descriptions of evidence use being concerned with the effectiveness of interventions or possible alternative priorities or approaches.

Policy studies scholars would not be surprised, however, that powerful stakeholders (or ‘vested interests’) end up shaping the understanding of evidence, or which pieces of evidence are championed as relevant for policy making. This is perhaps most clearly illustrated in how evidence has been presented or selected by the tobacco industry in relation to policymaking debates (c.f. Smith 2013b; Ulucanlar et al. 2014; Tong and Glantz 2007; Bero 2005). For example, tobacco industry-funded studies have been shown to have misrepresented the association between second-hand smoke and CVD (Tong and Glantz 2007), or the evidence in support of standardised packaging of tobacco products (Ulucanlar et al. 2014). Tobacco interests have also emphasised evidence in support of the economic contributions of their product, whilst questioning the evidence suggesting that policy interventions are needed to protect health (Smith 2013b). HIV/AIDS, on the other hand, touches on issues of gender and sexuality, drug use and sex work, which often leads to it being seen as a highly morally-contested issue. However these did not appear particularly relevant in Cambodia, serving as a reminder that we cannot assume a health topic will necessarily exhibit the same political characteristics in different settings, even if such features are commonplace in other cases.

Conclusions

In the three contrasting case studies of evidence use in health policy-making examined in this study, evidence types – and their framing – were found to differ greatly, despite them taking place in the same country setting. The findings reiterate past authors’ understandings that ‘evidence’ is not a uniform concept for which more is obviously better, or where a single model of ‘evidence based policymaking’ can prevail, but rather that different constructions and pieces of evidence become relevant given the politics involved in policy decisions, the nature of institutions involved, and the framing and conceptualisations of the issues themselves. Our comparative analysis helps to begin to trace out themes in linkages between the nature of contestation of health issues, the interests of established dominant stakeholders, and the logics by which those stakeholders operate – all of which work to shape which evidence is utilised or seen as policy relevant to inform health decisions. Whilst considerable further empirical research is needed in this area, this more nuanced understanding of evidence use may be of relevance to health policy-makers and others considering how to improve the role of evidence in health policy making.