Introduction

Apart from well-known multilateral and civilateral agencies such as the United Nations and Oxfam International and, at the bilateral level, national governments, a wide variety of players have started to look for opportunities to join the struggle against worldwide poverty and inequality (Develtere 2012; Richey and Ponte 2014; Schulpen et al. 2011). For different reasons, celebrities, private foundations, and companies have increasingly become active players in the field of international aid (Bishop and Green 2008; Cameron and Haanstra 2008; Develtere 2012; Develtere and De Bruyn 2009; Kinsbergen 2014; Richey and Ponte 2014; Samman et al. 2009; Yrjölä 2009).

Increasingly these new development actors become a subject of academic research. Interestingly, and indicative of their position in the field of international development, an omnipresent alternative actor in many northern countries remains largely untouched: the thousands of ordinary individuals who actively engage in the fight against poverty by starting their own small-scale, voluntary development organisation, independent of direct government support. Whereas other unusual suspects are increasingly being recognised as actors of significance both by practitioners and academics (see Richey and Ponte 2014), such ‘ordinary citizens’ are still underexposed in academia.

In order to partly close this knowledge gap, the present study focuses on the work of these citizens in the field of international aid by zooming in on the sustainability of their development interventions. From here on, we refer to them as private development initiatives (PDIs), defined as a group of people who offer direct and structural support to development interventions in one or more developing countries. What distinguishes PDIs from most of the established development actors is their small scale (i.e. annual budget and number of staff, with staff referring to both volunteers and paid staff members) and voluntary character (Kinsbergen 2014). Although PDIs are not unique to the Dutch context, the empirical part of this study focuses on development initiatives of Dutch PDIs and their local counterparts (Appe and Oreg 2019; Fechter and Schwittay 2019; Haaland and Wallevik 2017; Schnable 2015; Schulpen and Huyse 2017).

This study aims first of all to provide systematic insight into the sustainability of the development interventions of PDIs and, secondly, to look for determinants of interventions’ sustainability. We start our investigation by providing a brief background sketch of Dutch PDIs and mainly by arguing why it is worthwhile to take a closer look at their work and particularly the sustainability of their work. In the second part, we move to our understanding and operationalisation of sustainability and our data collection and methodology. We consequently present our findings following the distinction made between the lifetime and structural change dimensions of sustainability. In the subsequent discussion section, we start by explaining the limitations of the study and continue by comparing the findings with respect to sustainability of PDIs to other aid actors. We conclude by providing a brief overview of our findings and by discussing why PDIs and NGOs are in some ways comparable but also quite different.

Analytical Starting Points on PDIs and Sustainability

PDIs in a Nutshell

A study of nearly 900 Dutch PDIs (CIDIN PDI Database 2008–2009) shows them to be run by, on average, four volunteers, and to spend about 50,000 euro per year on development interventions worldwide, with a geographical concentration on the African (e.g. Kenya, Ghana, Uganda) and Asian (e.g. Indonesia, India, Nepal) continents. The majority of Dutch PDIs are founded and/or run by senior citizens (55 years of age and above) and established mostly as a result of interest piqued by a journey or longer stay in a developing country. Most of the PDIs are run completely on a voluntary basis. On average, PDI volunteers spend 37 h per month in their organisation (Kinsbergen 2014). As such, PDI volunteers spend nearly double the number of hours compared to volunteers in general in the Netherlands (Schuyt and Gouwenberg 2009; Van Herten 2009; Van Ingen and Dekker 2011). For the largest part of their budget, PDIs depend on donations from private donors found within their own networks, such as friends, family, or colleagues. In addition, many of them receive donations from institutional donors that co-finance interventions of PDIs (see Kinsbergen and Schulpen (2011) for a more elaborate background sketch).

We argue that PDIs are part of the development aid architecture and that they are thus susceptible to the same need for scrutiny of their work that befalls the bigger and more established bilateral and multilateral players and NGOs not only because PDIs are ‘by nature’ active in the field of development and development cooperation but also because some of the Dutch PDIs explicitly consider themselves to be actors in the field of international development and also want to be recognised as such.

Besides, PDIs may be small and voluntary but that does not mean that establishing and running a PDI is anecdotal; they are real and omnipresent. For the Netherlands, (debatable) estimates of their numbers range from 6400 to 15,000 PDIs. In 2014, 4% of the Dutch (adult) population was said to be actively involved in a PDI, which comes down to half a million participants (Plaisier and Schulpen 2014). In 2015, PDIs received contributions of about 47 million euro from Dutch households (Bekkers et al. 2017). In Flanders (northern part of Belgium), estimates on the number of PDIs vary from 1000 to 1300. The total budget of Flemish PDIs comprises about 63 million euros (Mevis 2016). In the United Kingdom, also, PDIs turn out to be a pervasive phenomenon. Whereas in England and Wales there were around 2,300 internationally operating charities with a budget up to 100,000 lb in 1995, this number rose to over 10,000 in 2014, of which nearly half had a budget of £10,000 (Clifford 2016). A European mapping in 17 countries concludes that although differently named and in different quantities, PDIs are common and widespread across Europe (Pollet et al. 2014).

But this ‘do-it-yourself’ type of development organisation is not limited to the European context. Of the more than 11,000 registered American relief and development organisations, nearly 60% (6600) have an annual budget smaller than $25,000 and hence are presumably run exclusively by volunteers or at least ‘bear heavy burdens of voluntary labour’ (Schnable 2015). In line with these numbers, Richey and Ponte (2014) point out that underexposure of these alternative actors in academia is especially critical considering the growing share of private aid flows in the total of development assistance. There is increasing importance for privately funded aid as offered by PDIs since traditional sources and therefore traditional organisations are more and more under stress (Richey and Ponte 2014).

Potentially this entails an imminent and dramatic reshuffling of the development arena, which historically has been dominated by a selective group of development actors, here referred to as the development establishment. Considering that NGOs, as part of this establishment, still play a part ‘but no longer form the central theme of development’ (Banks and Hulme 2012, p. 25) means that getting to know new and emerging (unusual) players becomes increasingly pressing.

PDIs often perceive themselves to be different from other players in the field of development cooperation, and they are often perceived as different by more established players precisely because of their small scale and voluntary character. Seen in a more negative manner, PDIs are often viewed by established development actors as amateurs. PDIs in turn view themselves as making an efficient and effective contribution to poverty reduction and, perhaps even more importantly, as organisations restoring the ‘human face’ of international development cooperation (Brok and Bouzoubaa 2005; Develtere and De Bruyn 2009). Convinced as they are that they are close to the local people and are able to reach the poor, they also expect to contribute to poverty reduction and even to be able to tackle the underlying causes of poverty (Brok and Bouzoubaa 2005).

This self-image is countered by critics doubting PDIs’ professionalism. Koch (2007), for instance, expects them to suffer from the same pitfalls as the established development actors have. ‘They see PDIs supporting orphanages without orphans, building schools without thinking of the teachers’ salaries or constructing wells that no local person is able to maintain. And they are seen as doing all this in a top-down, paternalistic manner. Their contribution to poverty reduction is then by coincidence at best and non-existing at worst’ (Kinsbergen et al. 2017, p. 224). Critical views have been expressed in several studies that since 2006 have analysed development interventions of PDIs. These studies show positive results as far as direct results of PDI interventions are concerned but criticise PDIs for their lack of context analysis, needs assessment, and evaluation and monitoring (Chelladurai 2006; De Bruyn 2013; Kamara and Bakhuisen 2008; Kinsbergen 2007, 2014; Schulpen 2007). Combined, these critical remarks lead to questions with regard to the prospective sustainability of PDI interventions.

The present study delves deeper into this sustainability-question by offering a first-ever ex-post assessment of 93 development interventions executed by 42 Dutch PDIs in cooperation with an equal number of local partners in four different countries. In doing so, the study adds to our understanding of this unusual development actor and thereby to our understanding of an important new strand in development cooperation and studies. It is especially these aspects, the sample size (93 interventions), the cross-country (4 countries) character, and the timeframe covered (1998–2014), that make the design of the study and therewith the insights gained unique.

Ex-post Sustainability

Since the 1980s, sustainability is, together with relevance, efficiency, effectiveness, and impact, one of the five yardsticks in the evaluation of development interventions (Brown 1998; OECD 1989). However, till today conducting ex-post evaluations, also referred to as post-closure or ex-post facto evaluations, is still not common (Napier and Hayman 2016). Most evaluations assess the sustainability of development interventions ex ante: they analyse to what extent development interventions meet preconditions for sustainability (Kinsbergen et al. 2017).

In a review of monitoring and evaluation experiences of development projects by major donor agencies in the 1970s, Binnendijk (1989, p. 210) described how efforts to evaluate longer-term results were often not completed since they exceeded the funding lifespan of the project. Ten years later, Stockmann (1997, p. 1768), while recognising that several ex-post evaluations had taken place, depicted how many evaluation studies still only ‘simply compile conjectures about the future sustainability of projects and programs’. Another decade later, Intrac comes to a similar conclusion stating that ‘few organisations are actually doing post-closure evaluations and there has been little shared information or learning to date’ (Napier and Hayman 2016).

Both in ex-post as ex-ante evaluations, two approaches towards sustainability can be distinguished. The first (widely adopted) approach offers a more formal definition and focuses on sustainability in terms of the lifetime of projects, programmes, or institutions. The relevant question is: do these projects, programmes, or organisations continue to exist after the withdrawal of external donor support (Brown 1998; DAC n.d.; Finsterbusch and Van Wicklin 1989)? Development projects or programmes are considered sustainable when they are able to deliver an appropriate level of benefits for an extended time period after major financial, managerial, and technical assistance from external donors is withdrawn (OECD 1989, p. 13).

Others see this formal approach as too narrow (Dietz and Zanen 2008; Edwards 1999; Fowler 2000; Stockmann 1997; Wilkinson-Maposa and Fowler 2009). As Brinkerhoff and Goldsmith (1992, p. 371) state, ‘just because a project, programme or organization endures, does not necessarily mean it is valuable’. The main concern should not be whether the project, programme, or organisation will last, but whether it makes a lasting impact on poverty (Edwards 1999). This second approach provides a more substantive analysis, with a strong focus on the extent to which interventions are aiming at fundamentally tackling structurally constraining factors that induce, maintain, or strengthen poverty and inequality (Edwards 1999; Fowler 2000; Stockmann 1997; Wilkinson-Maposa and Fowler 2009).

The distinction between these two approaches aligns with the division between Big D/little d development and the corresponding roles of development organisations (Banks and Hulme 2012; Banks et al. 2015; Hart 2001, 2010; Mitlin et al. 2007). As Lewis describes, although the origin of the ‘D/d’ distinction is found in historical analyses of development, it has also been applied by development scholars in several ways that go beyond this (2019, p. 1962). In this contribution, we follow Banks and Hulme (2012) in distinguishing between two different roles (service providers and advocates) of civil society organisations and link service delivery to Big D development and advocacy to little d development. In effect, the first sustainability approach assesses the extent to which development organisations are successful in their role as service providers (Big D development) in the long run; the second approach analyses the civil society function of development organisations and their contribution to long-term structural change (little d development).

In this study, we will combine both sustainability approaches. We analyse the sustainability of a development intervention by assessing first the extent to which the intervention continued to deliver benefits over an extended period, without external financial and technical support; we will refer to this as the lifetime of an intervention. The lifetime dimension of sustainability is assessed through three subquestions:

  1. 1.

    To what extent do the interventions of PDIs produce intended results in the long term at the level of output?

  2. 2.

    To what extent do the interventions of PDIs produce intended results in the long term at the level of outcome?

  3. 3.

    To what extent are output and outcome results achieved independently of the initial donor?

Second, we consider the extent to which an intervention aims to address and change the underlying causes of poverty; we will refer to this as structural change. By doing so, we will evaluate the role of PDIs as contributors to Big D and little d development and analyse to what extent interventions of PDIs contribute to sustainable development. The structural change dimension of sustainability is determined by asking:

  1. 4.

    To what extent do the interventions of PDIs contribute to structural change?

In classifying the intervention types of PDIs in light of this second dimension, we start from Korten’s (1990) four generations of strategies: (1) first generation, relief and welfare, are aimed at alleviating directly observable needs by service delivery; (2) second generation, community development, pursues the strengthening of local capacities of people to better meet their own needs; (3) third generation, sustainable system development, with a focus on the elimination of institutional and policy constraints; and (4) fourth generation, people’s movement, characterised by a vision of people-centred development with development organisations being facilitators of a global people’s movement.

The following example explains how the sustainability of the interventions will be evaluated. In 1998, a day-care centre was built to provide shelter and education for 75 street children between 4 and 18 years old. To assess the first sustainability dimension, the lifetime, we verify whether planned direct results of the intervention are still in place (output) and whether the planned objectives of the intervention are still reached (outcome). In this example, the questions are as follows:

  • Output: Is the centre still in place and in such a condition that it can be used for its intended purpose?

  • Outcome: Is the centre being used to provide shelter and education for the target group as planned?

  • Independence: Is the centre operating independently of the donor in management and financial terms?

The second dimension questions whether the intervention is aimed at targeting the root causes of the problem initially targeted. The question reads:

  • Is the centre mainly focused on taking care of those already in need by providing basic needs, or is the centre also running activities to prevent children from living in the streets?

Data and Methodology

This article is based on an analysis of 93 development interventions.Footnote 1 Those interventions were run by 42 local organisations and supported by an equal number of Dutch PDIs. This means that some local organisations were running numerous projects (e.g. primary school and kindergarten) or subprojects (e.g. construction of four classrooms followed by construction of a boys' dormitory), and some PDIs were involved as a donor in several interventions. These interventions were co-financed by the Dutch development organisation ‘Stichting Wilde Ganzen’ (from here on referred to as Wilde Ganzen). Founded in 1957, the organisation started to support small-scale development interventions implemented by Dutch PDIs and their local partners since the mid-nineties.

Four countries were selected for the research based on their relative importance with regard to the number of interventions supported by Wilde Ganzen in the past and present (and expectedly, the future): Kenya, India, South Africa, and Ghana. For India, the same sample of 10 interventions and partners that participated in an earlier study on PDIs participated in the current study (Kinsbergen 2007). For Kenya, Ghana, and South Africa, the selection started with a dataset of all interventions supported by Wilde Ganzen in the past: Kenya (292 interventions), South Africa (235), and Ghana (317).

The sampling procedure consisted of two phases. In the first phase, we applied a stratified sampling procedure based on the age of the intervention. This allowed us to see if and how the age of an intervention was affecting the sustainability of the intervention. With ‘age’ we refer to the number of years between the financial investment and implementation of the intervention (e.g. when the classrooms were being built) and the year the study took place (2014). Three broad age groups were defined: interventions with an age of about 5 years, 10 years, and 15 years were selected to participate. In addition, a small number of older interventions were included. From each of these age groups, interventions were selected randomly (for example, every third intervention). This resulted in a gross selection of 30 interventions in Kenya, 34 in South Africa, and 41 in Ghana.

In the second phase of the selection procedure, we selected the interventions to be included in the study. We considered the thematic focus of the interventions (e.g. education, health care), the type of intervention (e.g. orphanage, school building, hospital renovation), the budget (small versus large), and the type of partner (e.g. CBO, local NGO, school). By doing so, we ended with a heterogeneous sample of 93 interventions that reflected the variety of interventions and partners of PDIs. This sample allowed us as well to look for possible factors affecting the sustainability of the intervention. Table 1 presents the final sample for the four countries. See Appendix 1 for a detailed overview of the studied interventions.

Table 1 Sample
Full size table

The oldest intervention was implemented in 1990, the youngest in 2008. The average age of the selected interventions was 10 years. The average project budget was about 39,500 euros. The age of the local organisations in charge of the interventions varied from six to 52 years, with an average age of 22 years. All but three of the 42 selected partner organisations still existed at the time that the study took place, compared to 29 of the 42 PDIs. Twenty-two PDIs and local partners were still in cooperation. In these cases, the PDIs were either still supporting the intervention central to the study or started supporting new interventions of their local partner after that time. Figure 1 presents the thematic focus of the studied interventions. Forty-one of the interventions (43%) were aimed at improvements in the field of education (e.g. building classrooms or school buildings, furnishing a library). ‘Care and welfare’ type interventions (30%) formed the second largest group (e.g. renovating a play garden for a day-care centre, constructing a home for people with a disability), followed by 16 (6%) health-care interventions (e.g. constructing a clinic). There were a smaller number of interventions in the areas of water, employment, and agriculture. Three were income-generating interventions aiming to cover the running costs of an organisation or its interventions. The share of the different themes within the sample was in accordance with the results of earlier PDI studies and can therefore be considered representative (Kinsbergen and Schulpen 2011).

Fig. 1
figure 1

Thematic focus

Full size image

Data Collection and Analysis

During the first phase of the study, for each of the interventions, planned results as formulated at the start were traced back. This was done through a desk study based on available paper records and the digital project database of Wilde Ganzen and websites of PDIs and local partners when available. At Wilde Ganzen, paper records were only available for interventions supported since 2008. For each of the interventions and organisations, the background and origins of the intervention, the stakeholders involved, the planned output and outcome, and characteristics of the PDI and local partners involved were mapped. The planned output and outcome results as agreed upon were retrieved from the project database of Wilde Ganzen. It was against this light that the functioning and the results of the intervention as established during the field visit were analysed. For India, data from the Wilde Ganzen database were added to data collected during a field research in 2007 (Kinsbergen 2007).

During the analysis, both the principal investigator and a research assistant independently scored the interventions along the line of the two sustainability dimensions as discussed in Sect. 2.2. Based on the findings, for the output and outcome results, the interventions were divided into four groups: (1) output results were no longer being achieved, (2) output results were below expectations, (3) output results were reached as intended, and (4) output results exceeded expectations. For the financial independence, we distinguished interventions that were (1) dependent on one PDI, (2) dependent on several foreign donors, (3) dependent on both national and foreign donors and, finally, (4) those that ran in a self-sufficient manner. For the second dimension, the interventions were grouped in one of the four generation strategies of Korten (see above).

During the second phase of the study—the field research—the local partners formed the key source of information. During one- to two-day project visits, semi-structured (retrospective) interviews, based on an interview guide, were held with founders, managers, school heads, teachers, nurses, and as much as possible with actual and former beneficiaries of the projects. Field research with two partners (one in Kenya and one in India) was not possible, but these have been included based on written sources and discussions with key informants outside of the two partners. Anonymous processing of data was guaranteed to all respondents, meaning that no names or specific information that might allow for a clear link to existing organisations and interventions have been included in this article.

These data allow for assessing the sustainability of each intervention alongside the two-dimensional sustainability approach. Subsequently we looked for determinants of interventions’ sustainability: which factors determine the sustainability of the interventions? Building on Kinsbergen (2014), we analysed first of all how characteristics of the interventions and of the local partner organisations relate to the sustainability of the interventions:

  1. (i)

    characteristics of the intervention: do age, budget, theme, or type of intervention correlate with sustainability?

  2. (ii)

    features of the local partner organisation: do the experience of the organisation (measured by age) or the type of organisation (e.g. local institution such as a school, or religious institution such as a congregation of sisters) influence sustainability?

Although it is not the main aim of this study to compare the results of the four countries, in a more exploratory manner, we do evaluate whether the context wherein the interventions are embedded are an influence on the sustainability results.

  1. (iii)

    Context: does the location of the intervention (measured at country level) relate to the sustainability of the interventions?

Results

The Lifetime Dimension of Sustainability: Strong Results

The analysis of the interventions began with studying the current state of the intended output results—the direct results of an intervention. For each of the 93 interventions studied, it was observed and discussed during the project visit whether the planned output was still in place. Figure 2 presents the output results combined for the four countries in percentages.

Fig. 2
figure 2

Output results 2014

Full size image

We found that in 75% of all the interventions, the planned output results were achieved and still in place. In some cases, these were infrastructural interventions with a lifespan of more than 10 or 15 years. The boarding school mentioned in the methodology section was reconstructed in 1990. Today the school is still up and running, and the buildings are well maintained. In a small number of cases (n = 3), output results even surpassed the expectations. In 22% of cases, the output results were no longer there or deviated significantly from the planned results. For example, whereas one intervention planned to have six soup kitchens distributing soup to those in need, today only two are still in place. Table 2 presents the output results at the country level, both in absolute numbers and in percentages.

Table 2 Output results in 2014, per country
Full size table

Kenya had the largest share of interventions that were no longer functioning (17%) or whose output results deviated negatively and significantly from the intended results (25%). India, followed by South Africa, had the highest success rate according to output level; 97% and 86.5%, respectively, of planned output results were still met or even exceeded expectations. In Ghana, the current output results were below initial expectations in 35% of the cases.

The results of the previous section indicate that in a large majority of the cases, planned hardware investments were still in place. In this section, we focus on the outcome or the usage of the hardware: there is a school building, but is schooling taking place? In a similar manner as for the output results, planned outcome results were compared with the current outcome. Figure 3 presents the total results.

Fig. 3
figure 3

Outcome results 2014

Full size image

In nearly 80% of the cases, planned output results were still being met or even went beyond expectations; for the outcome results, this accounted for 69% of the studied interventions. This was, for example, the case with a secondary school. In 2004, the PDI supported the school with the construction of a school hostel. The school aimed to provide education for 300 children. In 2014, the school had a regional function and was still up and running, with more than 600 children attending classes and without overcrowding taking place. Nearly 20% of the interventions no longer reached the once-intended project goals. One of these interventions was an internet café, constructed and equipped in 2007. The infrastructure was still there, but after 2012 it was no longer being used. Table 3 presents the country-specific results of the outcome analysis, both in absolute numbers and percentages.

Table 3 Outcome results in 2014, per country
Full size table

Ghana had the highest share of interventions that did not meet the objectives as planned (47%). India (90%), followed by South Africa (59%), had the highest number of interventions that were still functioning as originally intentioned.

When adding up the output and outcome results, we first of all found that in nearly 65% of the cases (N = 60) both the planned output and outcome results as once planned were still being achieved at the time of the study. In the discussion section, we will shed light on the comparability of these findings to other sectors. The aggregated output and outcome results show as well that in 13 cases (8%), output results were still there as intended, but project goals were no longer being reached or were below expectations. This means that the constructed hardware (e.g. clinic) was still in place; however, it was not being used or was used in a way that significantly deviated from the original plan. South Africa had the largest number of interventions with output results being in place but not being used as intended. There were 19 interventions (20%) implemented by 14 different local partners where both output and outcome results were no longer being achieved or were below expectations. Ten of these interventions were in Kenya, six in Ghana, two in South Africa, and one in India.

Long-Lasting Dependency

The first dimension of the sustainability approach considered the lifetime of output and outcome results. The question, however, was not only whether output and outcome results were achieved in the long run but also whether these results were achieved independently of the initial donor. We consider here successively both the managerial and financial independence of local organisations and interventions.

In nine of the 42 local organisations, there was a foreigner, in most cases a representative of the Dutch PDI, (semi-) permanently present in the organisation. In six of these cases, these persons were actually in charge of the management of the organisation and its interventions. Local staff members supported them in their work. When asked what he would do if the Dutch director were to leave the organisation and therewith the development interventions, one local manager answered:

  • I would find a Dutch successor as soon as possible. (Interview with manager of a local organisation).

Three local organisations and their interventions were co-managed by foreigners together with local staff members. In three other cases, Dutch PDIs were running the organisation from a distance. Although in these cases there were no representatives of the PDI permanently physically present, they did exert a strong influence from the Netherlands. The local staff were mainly the implementers of the plans designed in the Netherlands. In 29 of the 42 local organisations, these local staff members could be considered the actual managers of the interventions.

Table 4 presents the financial situation of the interventions studied (in absolute numbers). At the time of the study, the functioning of 31 (33%) of the 93 interventions was still totally dependent on the financial input of one Dutch PDI. Nine interventions (10%) were managed with funds from several foreign donors, with or without their own contributions (e.g. from income-generating activities). Forty-one interventions (44%) had been able to diversify their donor portfolio further and also receive donations from several local donors. Twelve interventions (13%) could be considered self-sustaining, with most of them generating adequate income from the paid services they delivered.

Table 4 Financial situation of the interventions
Full size table

In all nine cases where foreigners were physically present, the influx of donations was strongly dependent on their efforts; people continued to donate thanks to their physical presence and active role in managing the interventions and the trust that this engendered with the donors. One of these Dutch managers explained:

  • Some of our main donors said they would cease to donate when I leave the organisation. (Interview with Dutch manager of a local organisation).

It was often these foreigners, who were in contact with donors (writing proposals, newsletters, answering e-mails) and who were the trustworthy face of the organisation, that convinced donors to entrust their money to the organisation. Their role also limited local staff members in developing certain skills (fundraising, communication) and limited donors in building trust in the local organisation and staff.

Nearly 40% of interventions achieving output and outcome results as intended or beyond expectations were financially completely dependent on one Dutch PDI or several foreign donors. In addition, in one out of five of these interventions, the management of the organisation in charge was not locally owned but in the hands of a foreigner, either physically present or an organisation at a distance. These results indicate that some of the successful results at an output and outcome level were there (partly) by the grace of the financial or managerial support of foreigners. More precisely, although the data do not allow us to say that these interventions would no longer have been there without the presence of these foreign managers or the support of these donors, there are strong indications that survival in the current state would have been challenging.

Structural Change: The Ripple Effect?

The second dimension of our sustainability approach questioned the extent to which interventions were contributing to structural change. We therefore studied the type of interventions local partners undertook and the longer-term effects of these interventions.

We found that all PDIs and local organisations started off responding to direct, observable needs or problems by implementing interventions around service delivery. The initial strategy undertaken by PDIs and their local counterparts can hence be typified as direct poverty reduction or first-generation type interventions (Korten 1987). Except for two of these, over time all of them continued to hold on to this strategy. Only two local partners transformed such a one-sided approach into a more comprehensive approach by including development strategies deliberately aimed at changing the structures and mechanisms in place that cause the problems they were trying to solve. They extended their strategy from direct poverty reduction with second-generation (community building) and third-generation strategies (sustainable system development) (Korten 1990).

Because of the above, it is to be expected that the majority of the interventions mainly changed individual or family lives. However, some local partners (and with them the PDIs) claimed to make a change that exceeds the life of one individual, as they expect to bring about broader, structural change. Many incorporated these broader results into their project goals. For example, by starting a school in a secluded area, one of the local organisations and the PDI involved expect to improve the life of the community as follows:

  • Building a school =  > increased level of education =  > increased job opportunity =  > increased income =  > increased living standard =  > increased school attendance of next generations =  > increased educated middle class =  > decreased corruption =  > …

Although this sounds plausible to a certain extent, the interventions these local organisations undertook were mostly limited to the first step of this cycle of change: addressing immediate needs by initiating service delivery-type projects. For most PDIs and their local partners, the underlying assumption was that this initial boost would initiate an upward spiral that followed a natural pathway, changing individual lives and also structures and systems at the meso (communities, regions) and even macro (national) levels. PDIs and their local partners have a strong belief in a ripple effect: their actual intervention is the water drop (e.g. the school building), and the ripples derived from this drop are the assumed, broader, more structural changes caused by this drop (e.g. increased job opportunity). Based on the results of the study and in line with Banks and Hulme (2012), we question this supposition. Providing basic needs does not automatically result in social change. Said differently, Big D development is not a warranty for little d development, and little d development is not a by-product of Big D development interventions.

Although the data for this study do not allow us to exclude any form of social transformation ignited by the studied interventions, there are several reasons to moderate expectations related to this. First, many PDIs and their partners support the same types of beneficiaries (e.g. street children) in a similar manner (e.g. shelter) over many years; this is indicative of the perseverance of certain problems and demonstrates that tackling the root causes of a problem requires different types of interventions. The persistence of symptoms over time proves their institutionalisation within social and political norms and problems (Hickey and Bracking 2005).

One of the studied interventions clearly indicates that it is not a matter of course that through interventions aimed at direct poverty reduction, broader and more structural change will take place. In 2003, a local Indian organisation implemented, with the support of a Dutch PDI, an intervention aimed at the reintegration of child labourers into formal education through a ‘cow project’. To compensate for income loss, parents participating in the project received a cow for each child they sent back to school. The project aimed to provide children with an education, allowing them in the longer run to find proper jobs and income to take care of their future families. During an informal community meeting, we talked to some of the former beneficiaries, mostly women. Most of them were in their early twenties, starting or already running their own family. By their account, their lives were comparable to those of their own mothers 10 years ago. Many of them were involved in low-paying home industries, such as the cigarette-rolling industry (‘beedie’), and others had left home to work in the textile industry; both are known to be extremely exploitative, low-paying industries. Child labour was still present in the village. They expressed the hope of finding a ‘proper’ job, for example as a nurse, but the required higher education was lacking, and suitable job opportunities were scarce. Although the project enabled them at the time to finish school, persistently restrictive structures and systems in their living environment prevented them from changing their living conditions more radically, which had been an overall aim of the project (Tables 5, 6, 7, 8).

Second, although a large majority of the local partners are aware of the mechanisms in place that result in poverty and exclusion, their interventions do not result from a strategy that explicitly includes and responds to these mechanisms. Almost all partners can formulate what could be referred to as a theory of change (ToC); a clear long-term vision defining the change they are pursuing and an elaboration of the strategy on how to realise this vision. In their daily practice however, their strategy is aimed at keeping their project running. Most of the time and energy of local partners is hence invested in maintaining or extending the actual project, as illustrated in the following example:

there are disabled children in need of care =  > we are taking care of them by building a home and arranging education =  > our main task is taking good care of the children by, among other things, looking for funds to provide for shelter, food, clothing, and education.

When these local organisations cooperate, for example, with the local government, it is mainly to convince the government to (financially) support their intervention. Support for the assumed trickle-up effect from Big D development to little d development is missing because of a lack of or very limited monitoring and evaluation by PDIs and their partners. Although many of them formulate project goals that include structural change, a large majority do not evaluate the intervention at that level. Consequently, they are not able to make statements on their actual contribution to social change, nor can they alter their strategy based on insights gained through the evaluation. Almost none of the PDI partners could go beyond ‘hearsay’ stories to explain what happened over time to the beneficiaries still participating in the intervention (e.g. increase in income) or to those who have left the intervention (e.g. job prospects).

Determinants of the Results: Significant Differences Between Countries

To understand the differences in terms of sustainability among the different interventions, we performed several analyses looking for determinants of the sustainability. Both output and outcome results and independence at the level of finance and management were included in the study. Since variation in the second dimension of sustainability (structural change) was too small, we did not conduct tests at that level. We first of all analysed whether characteristics of the intervention affect the output and outcome results. To determine this, we tested correlations between the output and outcome results and several background characteristics of the interventions (age, budget, theme, and type of intervention) and of the local organisations (age and type of organisation). We especially expected the age of the intervention to be a negative influence on its sustainability. Interestingly enough, our study does not find such an effect: compared to younger interventions, older interventions do not have a higher probability of no longer achieving the intended output and outcome results.

Secondly, we analysed whether the sustainability results differed between the four countries included in this study. Although it was not the research’s main aim to study inter-country differences, here the study shows very interesting results. We conducted several independent-samples t-tests. The results of this analysis show there are several significant differences between the four different countries at the level of output (see Table 1) and outcome (see Table 2). Overall, we can conclude that the results of the interventions in South Africa and India turn out to be the most sustainable, with India being the leading country and South Africa the runner up. Compared to the other countries, interventions in Kenya experienced the most challenges in terms of sustainability. A smaller number of interventions still achieved the intended output result at the time of study in Kenya compared to the studied interventions in India and compared to those in South Africa. Both these results were significant (Kenya versus India: t(52) = − 3.34, p = 0.00; Kenya versus South Africa: t(44) = − 2.06, p = 0.04). The results of Ghana were not significantly different compared to the other countries.

Table 5 Output results
Full size table

At the outcome level, a significantly lower number of interventions still achieved the intended outcome results in Kenya compared to interventions in India (t(52) = − 2.60, p = 0.01).

Table 6 Outcome results
Full size table

In Kenya, a significantly higher percentage of interventions at the time of the study no longer achieved the intended objectives. In addition, we found that, compared to interventions in India and South Africa, interventions in Kenya and Ghana were to a significant extent less locally owned due to both foreign financial and management involvement.

Table 7 Independence: financial management
Full size table
Table 8 Independence: management
Full size table

Because of the relatively small number of interventions no longer achieving their intended output and outcome results (n = 19) and the large variety among these (in terms of age of intervention, local partner organisation, budget, and theme), comparing the interventions in the different countries does not permit a statistically sound explanation of the differences found between the countries to be given. In addition, since an inter-country comparison was not the aim of this study, no specific country-level data have been collected that could contribute to a systematic, adequate explanation of the differences found between the four countries that participated in the study.

Looking at those interventions that no longer reached output and outcome results as intended, we find local government often plays an impeding role, varying from not living up to its promises to (financially) support the organisation in developing and running the intervention, to unexpectedly changing rules and regulations or changing government grant schemes. In other cases, the reasons for deviant results were due to developments in the intervention area. The realised output or intended objectives were therefore no longer adequate to respond to the changed needs of the population. An organisation changing its strategy from sheltering those in need to lobbying the government to take care of those in need turns a building once built as a refuge into its headquarters for its new role as lobbyist. In some cases, the cessation of development interventions was hence the result of changed insights or progress and should not be considered as a failure. Whereas in some cases the discontinuity of output and outcome results could have been prevented (by, for example, investing in a more diversified donor portfolio or in local capacities), other cases illustrate the uncontrollable reality in which most local organisations operate. They demonstrate that not everything is foreseeable and that success is not just a matter of good staff, organisation, and planning.

Discussion

Limitations of the Study

The study has some limitations related to the design of the study possibly affecting the internal and external validity of the study. All interventions included in this study received co-funding from Wilde Ganzen. The decision to co-fund these interventions was preceded by an assessment of a project proposal submitted by the PDI and/or its local partner to Wilde Ganzen. Because of this, the design of the study could possibly result in two biases. First of all, it could limit the generalisability of the results of the study related to the entire population of PDI interventions to those not going through a similar selection procedure. It could be assumed that because of the assessment procedure the interventions in our sample went through, the results of this study shed an overly positive picture of the sustainability of PDI interventions. Other co-funding organisations of PDIs use similar selection methods, so the bias due to being a partner of Wilde Ganzen (as opposed to other co-funding agencies) is considered limited.

Secondly, over time, the project evaluation procedure of Wilde Ganzen has changed significantly. More precisely, the number and type of criteria applied in the selection procedure has changed over the years. This, together with a varying number of project applications requesting co-funding from the organisation, has resulted in an approval rate varying from a bit over 40% in 2002, 28% in 2004, 20% in 2006 to 54% in 2008 (own calculations based on annual reports of Wilde Ganzen). This implies that not all the interventions in our sample went through a similar selection procedure that could possibly have affected the robustness of our results. However, since the age of the interventions turns out not to have affected the sustainability of the interventions, we do not expect this bias to have been effectuated.

Finally, the results of our study do not allow for an explanation of some of the differences in terms of the sustainability of the interventions in the different countries. To overcome this limitation and enlarge our understanding in this regard, in the design of similar future cross-country research, contextual differences and their possible impact on the sustainability of the interventions should be more systematically included in the data collection. Building on the results of our study, it would be, for example, interesting to collect data on the relation between (local) government and local civil society organisation in general and government support to CSOs in particular.

Comparability of Findings with Sustainability Levels of Other Aid Actors

For two reasons, putting the results of this study into perspective presents some challenges. First of all, this is because of the small number of ex-post sustainability studies of development interventions taking place. Secondly, those studies that do take place often study different types of development actors (established NGOs or government aid agencies) implementing different types and sizes of interventions (capacity building or community building) than those of the actor and interventions central to the study at hand. However, taking this into account, several similarities can be found between the results of this study and several other ex-post sustainability studies.

In line with the results of our study, Stockmann (1997) and Dietz and Zanen (2008) also found that the commitment and active involvement of donors positively influences the achievement of output and outcome results. Stockmann’s study included 15 interventions in 5 Latin American countries in the field of vocational training implemented and funded by the German government between 1962 and 1992. All of the training facilities constructed still existed (output) and were still in use as such (outcome). He described how the high standard of facilities can be attributed to intensive follow-up measures by the donor. This was also confirmed by the finding that during implementation organisational structures implemented by German experts often proved to be effective only as long as these experts were on location.

The study of Rijneveld et al. (2015), focusing on Benin, substantiates our reservations regarding the expected ripple effect. They studied to what extent over time a geographical ripple effect, as was expected in the theory of change of the intervention central to the study, actually took place. They found that ‘the metaphor of a seismic epicentre, where the intervention has catalytic effects and continues to spread until a tipping point is reached for the whole of rural Benin, does not do justice to reality’ (Rijneveld et al. 2015, p. V). Although not fully comparable to the ripple effect expected by the PDI initiators and their local partners, our study also found that catalytic effects are minimal.

Although studies like the ones mentioned in the previous paragraphs allow for reflection on the results of the study at hand, it remains difficult to compare the results of this study with other ex-post sustainability studies, especially when these studies concern other development actors. However, building on previous studies on PDIs and their development interventions, this study does allow a broadening of our understanding of the additionality of PDIs as development actors. PDIs are known for the long-lasting personal relations between them and their local partners and, sometimes, the (intended) beneficiaries. Many PDIs operate for multiple years in a same location; their partners are often small-scale community-based organisations or local institutions (such as a school or a clinic), and PDI representatives make frequent visits, often resulting in them being well known by the community. Their interventions often take place at a village or community level. In their people-to-people type of partnerships and way of intervening, most often there are no formal contracts that include an end date for the cooperation (Kinsbergen et al. forthcoming). Dietz and Zanen (2008) confirmed that these types of ‘bottom-up managed projects, with a long-term commitment, were valued most positively [by local community members]. The same applies for projects treating the population ‘with respect’. Building and maintaining mutual trust and long-term commitment were considered the most valuable quality of aid-providing agencies’ (154). This might, partly, explain the overall long-lasting continuation of development interventions and, with that, the results of this study. At the same time, this also explains the long-lasting dependency of local partners on the PDIs. The personal character of the relations and the strong commitment and involvement of the PDI members make stepping back challenging (De Bruyn 2011; Kinsbergen et al. forthcoming).

Explaining Long-Lasting Yet Non-transformational Interventions

We found that outputs and outcomes of interventions were not dependent on whether the initial investments took place 5, 10, or 15 years prior. In addition, we found that the commitment of the PDIs is also long-lasting. However, we also found that most activities were non-transformational in the sense that they contributed to little d development and not to big D development. How can this be explained?

First, the often-limited capacity of local organisations in terms of staff and money makes them invest all available resources in the day-to-day management of the intervention to the best of their ability. Tasks that take up most of the local managers’ and staff’s time include finding adequate funding and staff to keep the project going, writing project reports, dealing with local government regulations, and handling day-to-day troubleshooting. Because of this focus, there is little time or imperative to invest in, for example, monitoring and evaluating the project or translating actual, contextual developments into an up-to-date vision, policy, or project.

Second, those PDIs and partners that want to alter or extend their intervention strategy from direct poverty reduction to, for example, advocacy, or those that want to invest in monitoring and evaluation systems for learning purposes are often confronted with unwilling donors, both institutional and private. One of the local organisations studied started as an organisation offering education and practical support to street children. While expanding its activities with advocacy work and research, the organisation was confronted with declining support from Western donors, hampering its transformation. The impact philanthropist wants to make a difference and succeeds best when donating to clear-cut interventions with visible results over more abstract, long-term interventions (Duncan 2004).

This is not unique to PDIs and their local partners but is also a well-known challenge of established development organisations. However, adding to the complexity, private donors of most PDIs are often families, friends, or colleagues. The pressure to give in to their wishes, to demonstrate clear results within a fairly short time frame, and to preserve the organisation from failure makes the PDI more prone to opt for service delivery types of interventions. Kilbey (2006, p. 960) adds that small NGOs are especially vulnerable to what she calls ‘the erosion of values […] because the support and resourcing options they have are fewer’.

Not only are PDIs confronted with the wishes of their donors, but also the inner drive of PDI volunteers—investing on average 37 h per month in the organisation—makes the choice for clear-cut interventions with tangible results in the short term more evident. This is referred to as the ‘fun factor’ (Kinsbergen 2014; Kinsbergen et al. 2017). Many PDI volunteers mentioned that to keep going, they needed to see concrete results from the time, energy, and money they invested in the interventions. Related to this is what Salamon (1995, p. 46) refers to as ‘philanthropic particularism: the tendency […] to provide certain types of services […] based on volunteers' own particular interests and preference’.

Conclusions

In this study, we have analysed the sustainability of development interventions of small-scale voluntary development organisations. In doing so, we aimed to get a better understanding of this ‘unusual’ development actor and thereby further our understanding of an important new strand in development cooperation and studies. Although it is difficult to put the results at the output and outcome level into perspective due to a lack of comparative data, these results can be considered positive. In general, the target group (and in some cases even those that did not belong to the original target group) benefitted and still benefit from the interventions. In other words, small as they are, the interventions can make a difference in the lives of people and can continue to do so long after the initial investment. We can conclude that PDIs are successful in fulfilling their service delivery functions (Banks et al. 2015). This result should not be taken lightly, considering the character of both PDIs and their local partners.

However, these positive findings come with a catch, as many of these interventions are still strongly dependent on the support of one or several foreign donors. At best, this highlights the long-lasting relations between PDIs and their partners. At worst, it shows the continuing dependency of these interventions of outside support. Perhaps even more important is that the second dimension of ‘structural’ sustainability is especially at risk, with a large majority of the interventions having a unilateral focus on direct poverty reduction over time. Tangible development interventions are clearly preferred over a focus on systemic change via more activist approaches.

In looking at ‘ordinary citizens’ organising themselves to actively contribute to development, we find two distinguishing factors to be particularly influential on the intervention strategies of PDIs: first there is their strong dependence on private donors, and second there is their dependence on the unpaid time investment of PDI volunteers. Both are receptive to what could be called ‘charitable consumerism’ making the PDI donors and PDI volunteers want to ‘consume' charity: they are receptive and responsive to easier-to-sell problems and solutions and reluctant to support those interventions ‘addressing the complex structural difficulties of poverty and underdevelopment that a genuine development alternative would promote’ (Banks and Hulme 2014, p. 190). We therefore think the dominance of charitable consumerism in the hearts and minds of those initiating and/or supporting PDIs might explain their strong focus on long-lasting yet non-transformational types of interventions.