Governments and international organizations are investing resources to reduce preventable deaths and diseases across low-and middle- income countries (LMICs). Still, the World Health Organization (WHO) [1] reports that between 20–40 % of resources spent on health are being wasted. Inefficiency is caused by inappropriate use of medicine and equipment, medical errors, suboptimal quality of care, costly staff mix, unmotivated healthcare workers, and corruption [1]. Faced with these issues, program planners must make difficult decisions about the best ways to invest limited resources to improve healthcare services and population health.

In recent years, many governments, donors, consultancy firms and non-governmental organisations (NGOs) have started transforming the funding mechanisms of healthcare systems in LMICs, namely by implementing performance-based financing (PBF) to link payments to results. In this model, healthcare facilities are paid based on the extent to which providers meet pre-defined quantity- and quality-related performance targets, following an independent verification [2]. Examples of quantity-related performance indicators include the number of consultations for children under the age of five or the number of births per month. Examples of quality-related performance indicators include the healthcare center’s cleanliness or completeness of patient registries. Healthcare centers sometime have to reach a minimal quality score (e.g., at least 50 %) in order to be eligible for bonuses. Quality scores are also used as an inflator or deflator of bonus payments.

The implementation of PBF is rapidly expanding. For example, the World Bank reports that the number of African countries using PBF increased from four to 21 between 2006 and 2013 [3]. Despite the rapid implementation of PBF, it is unclear whether given the same amount of resources, PBF can buy more healthcare services or health than the status quo or other interventions aiming to strengthen the healthcare system in LMICs. Existing systematic reviews on economic evaluations of PBF mainly draw their conclusions from studies conducted in high-income countries (HICs) [4, 5]. The results of these systematic reviews therefore cannot be generalized to LMICs, seeing that contexts and resources differ significantly. Distinctive characteristics of LMICs may influence the relations between the costs of PBF and the outcomes observed in HICs. For instance, the initial fixed costs associated with building data infrastructure or monitoring systems may require different investments. According to Fritsche et al. [3], PBF programs tend to require about five percent of additional financing in Organization for Economic Cooperation and Development (OECD) countries compared to 30–40 % of additional financing in LMICs. Moreover, factors unrelated to the motivation of health workers or outside of their control may affect healthcare services to a greater degree in LMICs compared to HICs. On the provider side, these factors may be related to the lack of continuous training, drug supplies, tools and the availability of other resources. On the service-user side, these factors can be related to the difficulty of paying direct and indirect user-fees [6, 7]. Thus, it is important to evaluate whether PBF represents good value for money specifically within the context of LMICs.

The objective of this systematic review is to identify and synthesize the existing literature that examines whether PBF represents an efficient manner of investing resources. In line with Emmert et al.’s approach [4], pay-for-performance (P4P) was considered efficient when improved care quality or quantity is achieved with equal or lower costs, or alternatively, when the same quality or quantity of care was achieved using less financial resources.



Protocol and registration

We conducted a systematic review to identify and synthesize literature on economic evaluations of PBF in LMICs. This review is in line with the PRISMA statement [8]. The initial protocol was not registered.

Eligibility criteria

Inclusion criteria

In this systematic review, we included: 1) studies conducted in LMICs, as define by the World Bank [9]; 2) studies using experimental or observational designs to assess the costs (or inputs) and consequences (or outputs); and 3) studies in which a comparison between alternatives was made (including the status quo). We included studies that were primarily impact evaluations only if they also presented results on the costs of PBF. Following Drummond et al.’s [10] categorization scheme, we differentiated studies depending on whether costs, consequences, or both were considered. This approach results in a classification that distinguishes: “Type I” studies as full economic evaluations that make a clear connection between the costs and consequences of two or more alternatives (e.g., cost-effectiveness analyses, cost-utility analyses or cost-benefit analyses); “Type II” studies as partial economic evaluations that describe the costs and consequences of initiatives without making a clear connection between the two; “Type III” studies that compare the costs of the initiatives without providing an effectiveness analysis regarding the health services or health outcomes; and “Type IV” studies that provide information on the costs of a PBF initiative without any description of changes in healthcare services or health outcomes [4]. To avoid overlooking important literature, we included articles belonging to these four types of economic evaluation studies.

Exclusion criteria

In this systematic review, we excluded: 1) studies conducted in HICs, as defined by the World Bank [9]; 2) publications that did not provide empirical evidence, such as editorials and interviews; 3) non-comparative evaluations because full economic evaluations require the comparison of two alternatives; 4) studies that only described a PBF program or solely evaluated their effectiveness; and 5) studies that focused only on demand-side financial incentives, such as financial compensations or bonuses for people who seek healthcare.

Information sources

Searching in previous systematic review

We began our search by manually screening the reference lists of two recent systematic reviews to find economic evaluations of PBF focusing specifically on LMICs. A well-cited review, conducted by Emmert et al. [4], covered economic evaluations of PBF published between January 2000 and April 2010. The authors did not impose location-related restrictions. Meacock et al. [5] repeated the same search in September 2012 to ensure that no recent articles were omitted. We also screened the reference lists of additional relevant reviews that came to our attention during the search [1114]. By reviewing past systematic reviews, we were able to identify pertinent studies published between January 2000 and September 2012.

Searching in databases

As Rethlefsen et al. [15] recommend, we collaborated with a professional librarian from the University of Montreal. We adapted Emmert et al.’s [4] search strategy to find more recent literature on economic evaluations of PBF in LMICs. Our search differed from Emmert et al. [4]'s in that we: 1) added Mesh terms and descriptors to expand the search; 2) modified the list of search terms by using more truncated terms (e.g., “cost*” includes “cost-effectiveness”); 3) deleted currency-related terms (e.g., dollars, yen) to better target pertinent results, given the rapid expansion of PBF worldwide; and 4) updated the inclusion and exclusion criteria (see below).

We conducted electronic searches in two databases: PubMed and Econlit. Search limits included studies written in English and French, published between January 2012 and June 2014. These dates allowed us to have an overlap with the time frame covered by previous systematic reviews to avoid missing any pertinent articles [5]. The complete search history is available in Appendix 1.

In addition to the two databases listed above, we used Google and Google Scholar to identify other potentially relevant documents such as books, unpublished studies, study protocols, conference articles, and new PBF initiatives. We consulted the websites of governmental and scientific institutes concerned with PBF (e.g., the World Bank's website on results-based financing,; the Global Fund, We also contacted health economics experts to request information on additional ongoing or recently completed studies. We provided them with a list of the articles selected for this review and invited them to identify any missing article.

Study selection

One investigator judged titles and abstracts of potentially relevant studies according to inclusion and exclusion criteria (Table 1). When the investigator could not reach a final decision based on the abstract solely, she proceeded to review the full text. If a decision was still unattainable, a second investigator reviewed the article before reaching a consensual decision. Two investigators read and appraised the articles selected.

Table 1 Inclusion and exclusion criteria

Data items and extraction

Two members of the research team performed data extraction. The data extraction forms were custom-designed. The following information was extracted to summarize the articles: first author, publication year, country where study was conducted, characteristics of the PBF program, study objective (implicit or explicit), sample size, data gathering techniques, primary data analysis approach and main results of the study in relation to our focus.

Summary measures and data synthesis

The studies selected used a variety of principal summary measures (e.g., technical efficiency scores, Malmquist Productivity Index, difference in costs). Where possible, we present the effects of the interventions as the difference between the intervention and control groups at baseline and follow up percentages or scores. We could not perform a meta-analysis due to heterogeneity of studies and presentation of results.

Appraising methodological and reporting quality of included studies

We appraised the results of the studies by examining the relation established between the costs and consequences; the alternative interventions that were considered; the costs and consequences that were included or omitted; the study limitations; and potential conflicts of interests.

To help us synthesize our assessment of the overall strength of the evidence, we developed a concise list of questions, adapted from Drummond et al. [10].

  1. 1.

    Was a clear relation between costs and consequences demonstrated empirically?

  2. 2.

    Which types of designs were used to assess the effectiveness of PBF?

  3. 3.

    Were different types of interventions considered as alternatives?

  4. 4.

    Were the costs (or inputs) and consequences measured longitudinally to examine change over time?

  5. 5.

    Were all important costs (or inputs) and consequences considered?

  6. 6.

    Were the studies conducted in different countries and contexts?

  7. 7.

    Did the authors report potential conflicts of interest?


Study selection

In total, we identified 2, 639 potentially relevant articles throughout PubMed, Econlit, Google Scholar and Google. After eliminating duplicates and reviewing the remaining abstracts, 45 studies were retained for a more detailed analysis. Screening reference lists from earlier reviews and expert consultations yielded 8 additional articles. Thus, 53 full texts were assessed. Of these, seven studies met our inclusion criteria (Fig. 1) and were included in the review.

Fig. 1
figure 1

Search flow and results

Appendix 3 presents a list of articles that were screened, but then excluded. The most common reason for exclusion was that the articles did not focus on LMICs.

Study characteristics and appraisal

We present a summary of each study’s characteristics in Table 2. The table highlights the diversity of intervention designs, study methods and outcomes. We also provide a summary of our appraisal for each study in Table 3.

Table 2 Characteristics of included studies
Table 3 Appraisal of included studies

Synthesis of results and appraisal

The section below presents our overall assessment of the strength of the evidence, using the list of questions we adapted from Drummond et al. [10].

1. Was a clear relation between costs and consequences demonstrated empirically?

None of the included studies were classified as full economic evaluations that make clear connexions between the PBF costs and healthcare services and/or health (Type I). In other words, none of the studies included cost-effectiveness analyses, cost-utility analyses or cost-benefit analyses. For this reason, we classified the 7 studies as partial economic evaluations (Type II), as they described the costs and consequences of PBF initiatives without making a clear connection between the two. It is important to note that full economic evaluations are necessary to evaluate whether PBF provides good value for money in LMICs because they are more methodologically sound than partial economic evaluations [10].

2. Which types of designs were used to assess the effectiveness of PBF?

An intervention that is not effective cannot provide good value for money. Therefore, we examined the designs that were used to assess the effectiveness of PBF in the included studies. Of the seven articles, only one study reported using a randomized control trial to assess the consequences of PBF [16]. However, Witter and colleagues [14] have identified problems with the allocation to the treatment and control-groups for this study. It appears that some districts were found to have existing pay for performance schemes, requiring the allocation to be adjusted in a non-random way. This study found that the intervention group had an increase in institutional deliveries and preventive care visits, compared to the control group. However, there was no improvement in the number of women receiving any prenatal care; the number of women completing four or more prenatal visits; and the number of children receiving full immunisation schedules.Footnote 1 The other articles included in this review adopted a variety of observational designs, for instance, relying on difference-in-difference estimates, time series and trend analyses. The majority of studies did not use pre-intervention data in their analyses. Potential biases and mitigated results limit our confidence in the effectiveness of PBF programs, as presented in the studies.

3. Were different types of interventions considered as alternatives?

Economic evaluations require the comparison of two alternatives to identify which is more efficient [2]. Most studies in this review compared the implementation of PBF to the status quo. System-strengthening alternatives to improve the motivation of healthcare workers or service delivery were not used as comparators. Potential alternatives that could have been considered to test whether PBF provides the best value for money include: other funding mechanisms; monitoring (without financial incentives); providing performance feedback; training health workers; increasing leadership skills; encouraging collaboration; and fostering a culture that promotes trust and the intrinsic value of work [17]. In addition, more studies should attempt to tease apart the incentive effect from the resource effect. Only one study included in this systematic review increased the budgets of the PBF intervention and control groups by the same amount [16].

4. Were the costs (or inputs) and consequences measured longitudinally to examine change over time?

The seven articles examined the impact of PBF programs over different time periods. Gok & Altmdag [18]’s study ranges from 2001–2008; Bowser et al. [19] and Sabri et al. [20]’s study cover a four-year time period; and Zeng et al. [21], Basinga et al. [16], Rusa et al. [22], and Soeters et al. [23] report change over a two year period. From the studies in this review, little is known about how the relation between PBF costs and outcomes in LMICs evolves over the long term.

5. Were all important costs (or inputs) and consequences considered?

The studies did not provide a detailed description of the costs that were included or omitted. The studies mostly examined the immediate/direct financial costs and effects of the interventions. Authors generally did not attempt or were not able to quantify all the different types of costs and inputs (e.g., time and funds invested to monitor the delivery of health services, time spent filling out forms). Only aggregated costs were presented.

Overall, important effects on health outcomes and unintended consequences (e.g., reduction of healthcare services not rewarded financially) were not sufficiently considered.

6. Were the studies conducted in different countries and contexts?

The seven articles were conducted in only five LMICs. Table 4 presents the number of articles, the region and the income level for each of these countries. Many regions and countries currently implementing PBF are not represented in these studies [24]. Moreover, some countries like Rwanda are characterised by unique political contexts and demographic situations, limiting the generalizability of results to other countries.

Table 4 Countries classified according to the region and income level

7. Did the authors report potential conflicts of interest?

Six out of seven articles had at least one author that was or had been affiliated with an organisation involved in the implementation of PBF, thereby resulting in a potential conflict of interest. The interpretation of data or presentation of information may have been influenced by their personal or financial relationship with other people or organizations. Interestingly, only one author explicitly reported having been employed by an organization involved in the implementation of PBF as a potential conflict of interest [21].

Summary of the assessment

Only seven articles fit out inclusion criteria. Overall, the evidence of economic evaluations of PBF is weak for the following reasons: (1) none of the studies were full economic evaluations; (2) only one study used a randomized controlled trial, but issues with the randomization procedure were reported; (3) important alternative interventions to strengthen the capacities of the healthcare system have not been used as a comparator; (4) few studies examined the costs and consequences of PBF over the long term; (5) important costs and consequences were omitted from the evaluations; (6) very few LMICs are represented in the literature, despite wide implementation in these countries; and (7) most articles had at least one author that was affiliated with an organisation involved in the implementation of PBF, thereby resulting in a potential conflict of interest.


This systematic review highlights a lack of strong empirical evidence that supports the idea that PBF increases value for money in LMICs. This result is consistent with past findings [4, 5, 11, 14]. For example, a Cochrane review addressing the effectiveness of PBF in LMICs found that the current evidence base is too weak to draw general conclusions about the effectiveness of PBF in LMICs. Without reliable effectiveness-estimates, cost-effectiveness estimates cannot be calculated. Thus, it would have been surprising if this review had concluded differently.

The added value of this review is threefold. First, replications of past reviews are useful to validate results and find articles that might have been overlooked. Second, past reviews only included studies published up to 2011–2012. An update was therefore warranted, especially considering the rapid implementation of PBF in LMICs and the large number of studies that have published on PBF since then. Third, this is the first literature review with a search strategy that specifically targeted articles on the efficiency of PBF in LMICs. Thus, the current review has a different focus than past reviews, providing a collection of economic evaluations of PBF in LMICs that were not previously identified. For example, six of the seven studies in this systematic review were not included in the Cochrane review. Three of the studies were published after the Cochrane authors conducted their search [18, 19, 21]. The three other studies included in this systematic review, but not in the Cochrane review, were published and available in time to be considered [20, 22, 23]. However, they were not included and are not mentioned under “excluded studies” in the Cochrane review. Consequently, our systematic review may be useful to inform researchers and decision-makers specifically concerned with optimizing value for money in LMICs.

The reasons why so few PBF economic evaluations have been conducted in LMICs, despite wide implementation, is worth exploring. First, PBF is a complex intervention that targets multiple services. It is therefore difficult to evaluate the impact of PBF on health. Economic evaluations on this topic require complex modelling because diverse people and many conditions are affected. Second, it is difficult to obtain good quality cost data in LMICs because the information is not easily accessible. Last, international partners occasionally resist sharing their costs, usually substantial at start up. Promoting transparency may be useful to facilitate economic evaluations on PBF.

Strengths and limitations

While systematic reviews can take years to complete, this review was conducted within a few months to respond to timely concerns about whether PBF provides the best value for money in LMICs. The time frame usually required for producing systematic reviews has been found to be inappropriate for local policy makers that have urgent decisions to make [25]. This issue was highlighted by a decision-maker in Haiti, who widely shared an e-poster on the current results, claiming that “long publication delays would eliminate the important benefits of this review” (personal communication, June 13, 2015). Despite its rapidity, this review adheres to the core principles of systematic reviews in order to avoid bias and ensure rigor. A detailed description of the methods used was provided to promote methodological transparency, and to facilitate replication.

Our review has limitations. First, the studies varied in methodological quality and study characteristics. These differences made it difficult to adequately compare the results of the articles included in our systematic review. Second, as in the case with most reviews, our review might have suffered from publication bias. Sponsors of inefficient PBF programs may have blocked publishing to protect their interests [4]. Last, as with any review, we may have missed some relevant information during the selection and data extraction process.

Future directions

Future researchers and evaluators should attempt to make a direct relation between costs and consequences of PBF in order to draw conclusions about whether this financing option represents good value for money. There is a need to adopt stronger designs and to consider the long-term implications of these programs on costs and health outcomes. In addition, future studies should compare PBF to promising alternative interventions that aim to strengthen the healthcare system. It would also be beneficial to analyze the literature around PBF in LMICs using Drummond and Jefferson (1996)’s 38 defined quality criteria, as seen in Emmert et al. [4]'s systematic review, in order to generate an average quality score for each article.

During our search, it has come to our attention that at least three economic evaluations of PBF are currently being conducted in LMICs. Borghi et al. [26] published a protocol on the evaluation of a P4P program in Tanzania. Using a controlled before and after study, the authors aim to measure the cost-effectiveness of the P4P program. Moreover, two economic evaluations are being conducted on PBF initiatives in Malawi [27]. Together, these studies should contribute to the evidence on the efficiency of PBF in LMICs.


In contexts of limited resources such as LMICs, it is essential that funders and decision-makers aim to optimize the value obtained from the money invested in healthcare services, in order to address the pressing health needs of the population. Some stakeholders have proposed PBF as a promising avenue. However, this review has demonstrated that there is a lack of empirical evidence to support the claim that PBF represents good value for money. We still do not know if, given the same amount of resources, PBF buys more healthcare services or health than the status quo or other interventions. Full economic evaluations of PBF are needed to truly inform decision-makers in LMICs on how to make better use of limited resources to improve population health.