Background

Participatory budgeting (PB) aims to democratically allocate public money for local services, enabling communities to decide how public funds are spent [1]. It entails a multi-stage process, which typically concludes with a vote, where citizens deliberate among themselves and with government officials to allocate funds for public goods [2]. Since it’s origination in 1989 in Porto Alegre, Brazil, PB has been implemented in many countries across North and South America, Europe and Asia, including many high income countries [3, 4]. International institutions including the World Bank, the Organisation for Economic Cooperation and Development (OECD), the United Nations and the UK Department for International Development recognise PB as good practice [3, 5, 6] and worldwide PB schemes distribute multimillion dollars/euros [7]. In Porto Alegre, PB has been reported to increase public spending in the poorest areas of the city, reduce administrative costs and improve citizen participation [8].

Literature on PB abounds, but to our knowledge, there has been no systematic assessment of the impact of PB on health and wellbeing across the world. PB may influence health and wellbeing via several pathways: increase in democratic participation; greater community cohesion; improved allocation of funding to public services prioritised by the community; and increased access to medical services via increased spending on healthcare and social determinants of health such as sanitation or housing. Table 1 outlines the possible stages of how PB can influence health and wellbeing, informed by key literature [3, 6, 9, 10].

Table 1 How PB can affect the health, social, democratic and economic outcomes of individuals
Full size table

In order to establish the strength and reach of the existing evidence base for PB as a way of improving democratic participation, community cohesion, delivery of public services, and population health outcomes, we conducted a systematic scoping review of international literature to identify evaluations of PB schemes. Specifically, we examined what methods have been used to evaluate PB processes, what outcomes have been used to investigate the effects of PB, in which countries and at what geographic scale (e.g. neighbourhood, municipality, region). This scoping review is timely as there is growing interest from governments across Europe and beyond, in PB as a process for allocating public funds with the potential to address inequalities and increase community empowerment and cohesion [11, 12]. As PB is adapted and adopted in countries beyond Brazil, there can be less focus on social reform that was integral to the original process [13], with implications for benefits to health and wellbeing.

Methods

The methodology for this review was guided by recent recommendations for conducting scoping reviews [14, 15], the protocol is available [16].

Inclusion criteria

The review included studies of adults and children, as individuals or groups in communities involved in, or impacted by, PB. The intervention of interest was PB defined as communities deciding collectively how public funds are allocated. We did not include any form of individual budgetary decision--making (e.g. processes where individuals have personal management of welfare budgets, or employees have the opportunity to participate in defining their budget). Areas of interest included political, economic and budgetary, and health and social outcomes occurring at individual, population and systems levels. Political outcomes included outcomes at individual level such as participation, and systems level outcomes such as the democratic process at a local level. Economic and budgetary outcomes included outcomes at population level such as health or social service provision, housing, patterns of spending as the mechanism for impact on other outcomes, and provision of public goods (e.g. public parks, public safety). Health and social outcomes included outcomes at individual level such as health, wellbeing, self-efficacy, and empowerment, and at population level such as measurements of inequalities (e.g. poverty rates). We included any type of study design, quantitative, qualitative or mixed methods, with and without control groups or comparisons, which used primary analysis and reported on an evaluation of the PB system in relation to any of the outcomes listed above. This included peer reviewed published articles, books, reports and grey literature such as conference papers or working papers. There was no limitation on publication language or date; we did not find publications on PB prior to the 1980s when the process was established.

Literature search

Twenty-one electronic databases were searched in October 2016, with an updated search conducted in May 2018. The databases included peer reviewed articles and grey literature, and aimed to cover a broad range of health, social, political and economic literature, see Additional file 1: Table S1 for full list. No filters or terms were used to capture evaluation studies as evaluation studies are not indexed as such in bibliographic databases. No language or date limits were applied to the literature searches. Further articles from the review authors’ collections were also included. The search term ‘participatory budget’ was used as this is the relevant global label used for the process that we wanted to examine; the term is used internationally to identify this specific process. The search results were first screened by title and abstract, with 10% independently screened by two reviewers. The full text was then screened, with duplicate screening of 10%, and disagreements resolved by discussion and in consultation with the third reviewer.

Data extraction and collation

A data extraction template was developed in Microsoft Excel, tested and agreed by the review team. Duplicate data extraction was conducted on 20% of the included studies, see Additional file 1: Table S2 for data extraction template. As this was a scoping review, aiming to map the amount and type of evidence available on PB in relation to impacts on health and wellbeing, the literature was not formally appraised for methodological quality. The data were collated in summary tables and the results reported narratively.

Results

The literature search identified 1458 citations. After de-duplication and screening, 39 articles reporting on 37 studies met the inclusion criteria, see Fig. 1 for a flow diagram of the screening process. A summary of characteristics table provides details of the included studies’ study design, country, data sources, methods of analysis, outcomes of interest, and source of funding (Table 2), and Additional file 1: Table S3 provides further details of the studies. Most of the studies that met the inclusion criteria were published in English; three of the included texts were in Portuguese and one in Spanish. Several evaluations had been published in English and the language of the country of origin.

Fig. 1
figure 1

Literature screening flow diagram

Full size image
Table 2 Included studies: study design, country, data sources, methods of analysis, and outcomes of interest
Full size table

In which countries and policy sectors have PB processes been evaluated, and at what geographic scale?

The majority of the studies report on evaluations of PB in South America (n = 24), most in Brazil (n = 19), many (n = 7) focussed on Porto Alegre, where PB originated. The remaining South American studies were located in Peru (n = 3), Colombia (n = 1), and one multi-national study set in Brazil, Ecuador and Venezuela. There were seven evaluations of PB in Europe. One evaluation was conducted in the United States; three were set in Asia; and one study of PB was conducted in Africa. There was also one article collating case studies examining the impact of PB across twenty cities worldwide: South America (n = 12); Africa (n = 4); Europe (n = 2); Asia (n = 1); and the United States (n = 1).

The PB schemes operated and were evaluated at municipal level in 89% (n = 33) of the studies. The remaining four studies evaluated PB schemes at neighbourhood level [17,18,19,20]. The source of funding for the evaluation was reported in 59% of the studies; 38% government funding or non-government organisation, 21% reported an academic funding source. Academic researchers conducted 76% of the studies; the remaining 24% were non-government organisation evaluations.

What methods have been used to evaluate PB processes?

The included studies were split broadly by study methodology and the level of data used. The first category, randomised controlled trials and quantitative observational studies, lists studies that used modelling to identify the effect of the outcomes or analysed large-scale population level data (Table 2). The second category collates case studies that conducted primary data collection in the form of surveys, interviews and participant observation; some of these studies included descriptive analysis of municipal data. Almost half of the case studies (n = 10) used mixed method study designs (qualitative interviews or surveys and descriptive quantitative analysis). One of the observational modelling studies included qualitative methods [8].

Randomised controlled trials and quantitative observational modelling studies

We found one randomised controlled trial (RCT). This three-arm trial, randomised by region and district, assessed the introduction of PB in rural settlements in Russia. The trial compared introduction of PB assisted by administrative training, to training plus additional technical assistance from two fulltime consultants, and included control districts that received neither training nor consultancy assistance [21].

There were thirteen quantitative observational studies. Nine studies analysed population level data using linear regression techniques [8, 9, 22,23,24,25,26,27,28]; three studies used regression models to analyse individual survey data [29,30,31,32], and one study used cross tabulation descriptive analysis to assess population level data [33].

Case studies

Twenty-three evaluations were case studies providing descriptive analyses of single (n = 10) or multiple (n = 13) instances of PB. The methods used for these case studies included participant observation (n = 9) [34,35,36,37,38], interviews (n = 15) [18, 34,35,36,37,38,39,40,41,42,43,44,45,46], surveys of PB participants and PB officials (n = 14) [18, 19, 34,35,36,37, 39, 42, 46,47,48,49,50,51], or descriptive analysis of municipal data (n = 6) [17, 19, 20, 40, 41, 43], fifteen studies applied various combinations of these methods.

What outcomes have been used to investigate the effects of PB processes?

We define three broad categories of outcomes: political, economic and budgetary, and population outcomes (Table 3). The outcomes for each category are described below. While assessing the methodological quality of the studies was beyond the scope of this review, we include an indication of the results reported for interest (Additional file 1: Table S3).

Table 3 Cross-tabulation of study design by outcome category
Full size table

Political outcomes

Outcomes we labelled as ‘political’ were reported by two thirds (n = 26) of studies. These outcomes included: participation in PB by disadvantaged sections of the population; impacts on further involvement in political or civic activities; and learning about political processes as a result of engagement in PB. The majority (n = 20) of political outcomes were evaluated by descriptive case studies. Thirteen studies examined participation in PB of disadvantaged groups including women, people with low income, low education, disability, or of an ethnic minority, with overall mixed results reported. Several observational studies reported some increase in participation [8, 21, 22, 29, 32]. The case studies reported more equivocal findings, levels of participation by disadvantaged groups was increased [37], low [20, 47] or mixed [41]; barriers remained [43] and active participation (i.e. taking part in discussions) was found only to increase with increased years of involvement in PB [38, 50].

The relationship between PB and further involvement in political or civic activities was explored in six studies [30, 31, 34, 35, 37, 42, 51], sometimes reporting that the individuals engaging in PB were already involved in other civil society organisations. Changes in how people acted in political arenas as a result of PB were examined by nine studies [18, 30, 32, 36, 39, 42, 44, 46, 48], tending to report positive impacts. Learning about political processes as a result of engagement in PB was examined by four case studies [19, 37, 42, 52], in general reporting some increased learning of political processes.

Economic and budgetary outcomes

Twelve studies reported on economic or budgetary outcomes such as local tax revenue or local government spending on public services such as healthcare, sanitation and housing. Two studies analysed impacts on local tax revenue from municipal records [21, 24], using fixed effect regression analysis to examine whether there was a change in local tax revenue according to the implementation of PB, reporting positive results.

Thirteen studies examined the allocation of PB funds. Three studies set in Brazil assessed spending on healthcare and sanitation, comparing municipalities that implemented PB with comparable areas that did not, reporting positive results [9, 25, 26]. All three studies applied linear regression analysis to large datasets: for all Brazilian municipalities with available data [25]; and for municipalities in Brazil with a population of over 100,000 inhabitants, 220 municipalities from 1991 to 2000 [9] and 253 municipalities between 1989 and 2008 [26]. Analysis of PB in Indonesia found areas with more low income households were less likely to benefit from the PB process [28]. Linear regression was also used to examine allocation of PB funding in relation to citizens’ priorities in Russian settlements [21]. The impact of PB on the number and quality of agricultural services provided in 100 randomly selected Peruvian municipalities was assessed using linear regression analysis of data from the municipalities and central government [27]. The impact of PB on the provision of water services in 219 districts in Peru was analysed using linear regression, reporting no improvement to the services [23]. Budgetary expenditure on public services including water and sanitation in Porto Alegre in Brazil was analysed using difference in difference regression analysis, reporting positive results [8]. Municipal budgetary and census data from the Brazilian Institute of Geography and Statistics was used to conduct a descriptive analyse of the allocation of PB funds according to level of deprivation, finding mixed results [33]. Four further studies examined which public services received funding from PB, which included provision of recreational public spaces [17, 20, 49], and micro credit centres [40].

Health and social outcomes

Health or social outcomes, such as infant mortality, poverty rates, education, and access to sanitation, were evaluated by eight studies, most studies examining multiple outcomes. The impact of PB on infant mortality was examined by five quantitative observational studies located in Brazil [8, 9, 24,25,26]. These studies report mixed results. The two more recent studies reported that PB reduced infant mortality [25, 26]. Poverty rates in municipalities implementing PB were assessed by two studies in Brazil, both of which reported reductions in poverty [8, 9]. Education, measured as either child or adult literacy or years of school attendance, was investigated by two quantitative observational studies and one case study, with mixed results [9, 18, 24]. Access to sanitation, piped water supply and sewerage, was measured by one observational study and one case study, reporting positive results [8, 39].

Discussion

This scoping review provides a systematic analysis of studies evaluating health and wellbeing impacts of PB. We identified 37 evaluation studies focussing on issues relating to health and wellbeing or delivery of public services, a relatively small number in comparison with the estimated 2000 to 2700 PB schemes implemented worldwide [11, 53]. While a few evaluations used population level datasets to examine health outcomes in South America, overall there was limited scope in the methods used to assess PB, and the vast majority of studies focused on political outcomes.

The majority of studies found were single or multiple case studies, describing the scenario of individual PB schemes. There were fewer observational quantitative modelling studies using large population datasets, and only one study that combined this with qualitative analysis. This dearth of mixed methods approaches is somewhat puzzling. PB interventions are intended to realise a complex range of democratic and social goods through both the processes and outcomes of public participation in budgetary decision-making. Mixed methods approaches, with their capacity to combine exploratory and explanatory research designs [54, 55], are uniquely suited to conduct multidimensional evaluations of both processes and outcomes [56]. While further detail of mixed methods may be included in book length descriptions of PB processes (e.g. [57,58,59]), our review suggests that there is considerable scope for methodological development and innovation using mixed methods approaches to evaluate PB.

The majority of studies identified in this review were evaluations of PB in South America. All of the quantitative observational studies analysing population level data were conducted in South America, with the exception of the randomised controlled trial conducted in Russia. Evaluation of PB in Brazil is possible due to the availability of population data aggregated at municipal level. Such data is not routinely available in many countries [60]. Robust evaluation of PB in other countries may require developing databases at local government level. The concentration of PB evaluation in Brazil has strong implications for policy makers in countries beyond South America interested in implementing PB. As PB has been developed in other continents, and in high income countries, the processes, budgets, and scale of the initiatives have become more varied [13, 61]. Also, the rationale of social justice underpinning PB when it began in Porto Alegre has often become marginalised in PB outside Brazil [13]. While the results of PB in Brazil may be generalizable to some other countries with similar levels of existing public services, in general, these differences suggest that the results of evaluations on health and wellbeing outcomes found in PB in Brazil may not translate to PB in other contexts. The main focus of PB evaluations was on political outcomes, we found less evaluation of outcomes relating to delivery of public services or assessing health outcomes or wellbeing outcomes related to poverty. In part, this may relate to who is conducting the analysis; as far as we could determine, the majority of studies were conducted within the fields of political science and public administration. Despite worldwide implementation of PB, this review finds that the implications for health and wellbeing have not been the focus of attention in public health literature. In our review, the exception is provided by Boulding and Wampler [9], who discuss the possible impacts on wellbeing, finding limited research in this area. More recently, studies have accessed population level datasets to investigate health impacts, focussing on outcomes such as infant mortality, education and poverty rates (e.g. [25, 26]). However, the limited evaluations of PB health and wellbeing outcomes may also relate to the ad hoc quality of many PB processes. Only Brazil has institutionalised PB to the point where comparative and longitudinal evaluations become viable, which accentuates the challenge of assessing health and wellbeing outcomes. Nevertheless, recent policy developments in other countries (e.g. Harkins et al. [12, 13, 62]) are seeking to embed PB in local government for the long term. This offers the opportunity for a PB evaluation agenda that transcends the geographic and thematic foci prevalent in the field.

Strengths and limitations of this review

We used systematic, transparent methods with predetermined inclusion criteria. It is possible that some articles may not have been identified in the literature search, however, we are confident that the search of twenty-one databases provided us with a broad sweep of international peer-reviewed articles and grey literature on PB. As this was a scoping review, we did not make a formal assessment of the risk of bias in the included studies. Our aim was to map out what evaluations relating to health and wellbeing have been conducted. This section outlines key learning points and implications, in particular regarding the methodological, geographic, and thematic foci that dominate the field of PB evaluations.

Conclusion

The findings of this review lead us to recommend further evaluation of the impact of PB on health and wellbeing be conducted in a range of national contexts. The lack of substantial evaluations outside Brazil, and the variations of PB being implemented worldwide, support the recommendation that when PB is being adopted, adapted and initiated, this should be accompanied by rigorous evaluation of the process and expected outcomes, using appropriate comparators. Governments involved in supporting and developing PB processes are uniquely placed to ensure that evaluation is not an afterthought, but an embedded component of robust PB policy over the long term. PB has spread globally, partly on the basis of claims regarding its potential to empower communities and improve people’s lives. Our review indicates that the PB field needs a stronger evidence base in order to substantiate and refine those claims in a variety of contexts.