Countries and governments are divided on the fora and mechanisms for agreeing on effective development cooperation. But progress can be made on measuring what countries are doing in different areas of cooperation. This chapter looks at how we can assess the quality of development cooperation. It sets out a framework for measuring development cooperation across three areas: development finance; country policies affecting the exchange of goods, people, ideas, and capital; and global public goods. It considers the availability of measures against that framework and concludes on how these measures can be developed, or where new measures are needed, to provide a holistic assessment of development cooperation.
- South–South cooperation
- Global public goods
It is clear from other chapters of this book, in particular that of Gerardo Bracho, that the mechanisms for effective development cooperation between countries—either through the Global Partnership for Effective Development Cooperation (GPEDC), the Organisation for Economic Co-operation and Development (OECD), or the United Nations (UN)—are currently uncertain, or contested. But progress can be made on measuring what countries are doing in different areas of cooperation. One of the major constraints Bracho highlights to deeper collaboration relates to whether all countries have “the same responsibilities as traditional donors” in the effectiveness agenda and whether each should be “subject to practically the same scrutiny” (Bracho 2021, Chapter 17). However, even before we consider respective countries’ responsibilities and scrutiny, we can at least try and measure the contributions of different countries to development on a consistent basis. Such a measurement would have value in a number of dimensions. First, it would enable countries to understand what they are contributing to development in absolute terms. Second, it enables comparison and learning between different approaches. Third, it can give a measure of the potential for improvement. Fourth, it can enable scrutiny of these areas to drive improvement, and finally, it can provide the underlying measures to enable contributions to be assessed against responsibilities. This chapter aims to take stock of our ability to measure contributions to effective development cooperation across different countries.
What do we mean by “development cooperation”? The principle I use here is that “development cooperation” refers to a country’s policies and how these affect the current and future welfare and growth of other countries’ people and economies.1 So, even countries that do not have an explicit policy towards other countries are included because their policies—for example, on climate, migration, and trade—have a bearing on people elsewhere, regardless of their intent. In this respect, all countries are global citizens and have a development impact that can be measured. This is consistent with the universality (i.e. applying to all) of the Sustainable Development Goals (SDGs) and the 2030 Agenda for Sustainable Development, which sets the framework on the goals, targets, and measures applying to, and agreed on, by all countries.
In order for development cooperation measurement to be meaningful, it needs to be undertaken with reference to the “size” of the country being measured. Taking greenhouse gas emissions as an example, it is clear that—all else being equal—a more populous country will emit a greater absolute volume, so measuring that country’s effort will mean calculating a per head figure. Similar arguments can be made for “aid”, which is measured relative to the size of the economy. Using measures relative to country size can enable comparisons between countries at different income and population levels. Of course, this means that absolute contributions of countries are ignored. So, although the United States is the largest absolute aid donor, its government gives relatively little compared to the size of its economy. The absolute contribution (see below) is what matters for global impact, but when it comes to the measurement and comparison of efforts, this must be scaled by an appropriate measure of the relevant size of the country.
Differences in citizens’ footprints in individual countries notwithstanding, the largest (often the most populous) economies are likely to have the largest absolute spillovers on other countries—for that reason, their development cooperation should be of greatest interest to us. If the level of a country’s income means it should take a greater responsibility for its impacts, then the top right of Fig. 11.1 illustrates those countries of greatest interest.
The chart demonstrates that the G20—formed broadly on the basis of the size of members’ economies—is still essentially the largest economies,2 and that the vast majority of countries with higher incomes are predominantly members of the OECD. Together, the G20 and OECD countries produce around 90 per cent of global GDP, and likely a similarly substantial portion of transboundary spillovers.
Note that we do not consider the actions of private actors in this analysis. This is not that the private sector is unimportant, but rather that country governments set the framework for the private sector and—in assessing development cooperation—we are concerned with what governments do, or do not do. However, the basic principle we discuss should also apply to companies—they should understand and measure where their activities in the market lead to spillover impacts on others outside the market. This would include energy companies quantifying their impacts on the climate, and tobacco companies quantifying their impacts on health. All companies should understand these impacts, as they are responsible for them. A promising initiative in this regard is the Global Reporting Initiative (2018), which helps to develop common standards for sustainability reporting. In my view, every company creating material spillovers should quantify these impacts in a move to “quantified corporate social responsibility”.
But why do we need to measure development cooperation when we have the targets and indicators underpinning the SDGs? The 2030 Agenda and the SDGs are the globally agreed framework around development, but they are not focussed on what one country does for another, and they offer only a partial picture of development cooperation. The 17 SDGs are supported by targets and quantitative indicators. However, the clear majority of SDG indicators relate to national performance rather than transboundary or spillover effects (see Table 11.1). Of the 244 indicators supporting the SDGs, fewer than 30 per cent measure a “transboundary” effect. Even SDG 17 (Strengthen the means of implementation and revitalise the global partnership for sustainable development) has over a third of its 25 indicators focussed on national performance. The OECD has undertaken a similar exercise, focussing on the 169 targets in the SDGs. It finds that 169 (57 per cent) can be described as having transboundary elements (Organisation for Economic Co-operation and Development [OECD] 2019). These indicators and targets therefore provide only a partial picture of development cooperation.
So what can we measure on development cooperation now, and what more do we need? The remainder of this chapter looks at the scope of what we should measure, considers the state of the measures in each area, and concludes with where future collaboration is needed.
2 Framework for Measuring Development Cooperation
Here, I consider development cooperation in three different, perhaps overlapping, areas,3 which I think is helpful conceptually. First, development finance—including aid and concessional finance—can be provided by a government to another country to support its development. Second, policies with bilateral international impacts—in particular those related to the exchange of goods, capital, people, and ideas (such as trade and migration policy)—have impacts both domestically and in other countries, often with mutual benefits. Third, countries’ actions contribute to, or detract from, global public goods (GPGs) (or regional ones)—in particular, on climate, but also in areas such as security. My colleague at CGD Charles Kenny has generated an illustrative (rather than exhaustive) list of 72 GPGs and examined their spatial and temporal distribution (Kenny et al. 2018) (Table 11.2).
The distinction between these three groups relates to the focus of the impact. The first two groups both relate to policies that bilaterally affect other countries. The first group—development finance—is broken out separately, as often its intention is to (positively) impact another country. In addition, development finance is often seen as a critical limiting factor in development and receives significant attention in development and the literature. The second group relates to policies that have a direct effect on a partner country and may or may not have a positive effect or mutually beneficial effect. The third group has impacts—intended or otherwise—that are international but affect several countries simultaneously, or have the characteristics of public goods or bads (i.e. countries cannot be excluded from their impacts and cannot diminish them). Table 11.3 elaborates on these descriptions with the domestic and international impacts.
We now consider each of these areas, in turn, and consider the most important areas to measure.
3 Development Finance, Aid, and Measuring Quality
In this section, we look at (i) measures of quantity of concessional aid/finance, (ii) what we know about effective development finance and, (iii) what measures we have available on quality.
3.1 Measuring Concessional Development Finance and Aid Quantity
Measuring the quantity of aid—or concessional development finance—requires a common definition. Most efforts and our starting point are to concentrate on the (net) concessional element of governments’ grants or loans to other countries. Calculating this requires data that is consistent between countries and a common methodology to establish how concessional the lending is. For the 30 members of the OECD’s Development Assistance Committee (DAC), both of these exist in defining official development assistance (ODA) and “other official flows”, though the methodology remains contentious and arguably does not provide a consistent guide to the concessionality of development finance (OECD-DAC 2018). Still, the original aim of the DAC—to agree on standards in order to enable a fair comparison between countries of different economic sizes in supporting development and provide countries with good incentives to do so (Scott 2015)—remains highly relevant.
Beyond the DAC, there is no common data nor an agreed methodology on calculating concessional finance. There are valuable efforts to consistently and comprehensively measure all financial flows—in particular, the task force on Total Official Support for Sustainable Development (OECD-DAC, n.d.). In February 2019, the task force published “emerging reporting instructions”—so usable measures are therefore some way off, and in any case, they would not identify the degree of concessionality in that finance and may not be available for all countries.
To calculate an estimate of government concessional finance provided by each country, there seem to be several possibilities. First, we could use government budgets to calculate the cost—to the taxpayer—of a country’s aid and development finance efforts.4 This would reliably estimate the financial effort a country is making to support development, but it relies on there being publicly available information—suitably disaggregated accounts—on a country’s overseas activities. Second, an important element of concessional finance is the funds provided to international development organisations. This could be limited to those “multilateral” institutions with open, accessible accounts or, ideally, also include bilateral agencies—see McArthur and Rasmussen (2018), for example. Third, countries could be surveyed to provide their own estimates of concessional finance. Fourth, if government development finance, and its terms, could be measured, the degree of concessionality could be calculated or estimated (AidData produces figures for China on this basis). A new paper by me and my colleagues combines these latter three approaches to generate a new measure of “Finance for International Development” which enables more consistent comparisons across countries (Mitchell et al. 2020).
On the “quantity” of concessional development finance then, there seem to be some possibilities for making estimates. These will produce imperfect or incomplete figures, but these estimates will perhaps incentivise countries to do better, or ideally to come together and agree on common definitions.
3.2 What Does Effective Development Finance Look Like?
It is clear from evaluation evidence that aid and development finance can be transformationally positive or, conversely, completely wasted, or even damaging. Waste can occur in spite of good intentions, where interventions turn out to be less effective at reducing poverty than expected, as was the case with microfinance (Roodman 2012, p. 2), or because effectiveness is subordinated to geopolitical or commercial interests, for example in the famous case of the Pergau Dam (Lankester 2012). Aid might also have unintended effects: there is evidence that American food aid prolongs civil conflict (Nunn and Qian 2014). However, aid can also have enormous positive effects. In just under 20 years, Gavi, the Vaccine Alliance says it has treated 0.7 billion children and prevented 10 million deaths, generating savings of $18 in healthcare costs, lost wages, and lost productivity for each $1 spent (Gavi, the Vaccine Alliance, n.d.).
Countries discussed “aid effectiveness”—specifically the high-level principles and practices by which aid is allocated—and reached agreements in Rome (2003), Paris (2005), Accra (2008), and Busan (2011) (OECD, n.d.-a). Over the past decade, the context altered significantly with, for example: the agreement of the SDGs; the shift in the world’s major economies and concentrations of extreme poverty; and aid providers changing the instruments they use. The principles (OECD, n.d.-b) agreed at Busan were:
Ownership of development priorities by developing countries: Countries should define the development model that they want to implement.
A focus on results: Having a sustainable impact should be the driving force behind investments and efforts in development policy-making.
Partnerships for development: Development depends on the participation of all actors and recognises the diversity and complementarity of their functions.
Transparency and shared responsibility: Development cooperation must be transparent and accountable to all citizens.
Busan also gave birth to the GPEDC.5 The GPEDC now collects data with indicators6 grouped under these four principles. The GPEDC’s online platform enables comparisons between “development partners”, but for some countries, the data is thin, or non-existent (e.g. in the 2017 results, China scored 100 per cent for indicator 1a—“proportion of new development interventions draw their objectives from country-led results frameworks”—but this is based on just one country’s response). The GPEDC does not attempt to aggregate the indicators into themes, nor overall scores; nor does it rank agencies or countries. We return to measures that do attempt to compare, combine, and rank below.
What does research say about the impact of development finance and the practices that enhance its effectiveness? Much research focusses on GDP growth as the variable of interest, although it is clear that much aid is not targeted at that outcome—in particular, humanitarian aid, which covers 12.8 per cent of aid for DAC countries (OECD, n.d.-c), is most needed where economies are shrinking. Still, in the case of increased levels of education and health, we would expect some positive impact on GDP, and perhaps regression-based analysis can provide insights on effective aid. Howarth (2017) provides a good overview of this literature and concludes:
[T]here is very little evidence to support the “hard” sceptical view that aid actively harms growth. It is, however, now understood that aid is subject to diminishing returns, and increasing it beyond a certain proportion of a recipient’s GDP may have a harmful effect. Expectations about what aid can achieve have also become more realistic.
We are currently undertaking a literature review on the evidence on the determinants of effective aid that will consider issues such as using (recipient) country systems, recipient ownership, predictability, and transparency as traits of aid associated with higher impacts. Similarly, the evidence is clear that tying aid—that is, requiring aid to be spent only with providers from your own country—reduces its effectiveness,7 but it still features prominently in aid providers’ commitments (Meeks, n.d.).
Which of these principles or practices can and should be measured? And how important are they? Below we move on to efforts to measure and aggregate measures of quality that attempt to bring these together by donor or agency.
3.3 Data Sources for Measuring Development Finance Effectiveness
The evidence and theory about aid effectiveness is all very well, but what can we actually measure?
There are just three main sources that can be used to measure elements of development finance quality consistently across a wide range of countries (OECD-only measures are discussed below). These are:
GPEDC survey—collected 10 indicators under four themes8;
Listening to Leaders survey—AidData’s (2018) survey has been done twice, 2017 and 2014, and it provides data on the level of “helpfulness” and “influence” for providers of development cooperation;
The International Aid Transparency Initiative—which is a standard for open data publishing that is available to all countries as well as non-state donors, and it enables some analysis of a large proportion of aid.
We have already seen that the information on the quantity of concessional finance is limited, and that these sources provide a relatively limited picture of the quality of development finance.
The GPEDC survey measures indicators of development finance effectiveness but, as noted above, the results are dependent on response rates. In the 2016 round, this led to patchy coverage, at least in terms of being able to assess some major countries providing assistance—for example, some of China’s results were based on just one response (we are awaiting the details of the 2018 results). In addition, many Southern providers are averse to efforts to define, monitor, and compare development cooperation measures for both political and technical reasons (Besharati and MacFeely 2019). AidData’s (2018) Listening to Leaders survey measures the views of leaders in low- and middle-income countries, but it is not a direct measure of aid effectiveness. The International Aid Transparency Initiative, which also feeds into the GPEDC’s monitoring framework, improves transparency by hosting a machine-readable database of aid projects, and it gives cross-country comparisons of transparency in its published statistics. Publish What You Fund used this, along with other information, to create the Aid Transparency Index (Publish What You Fund 2018), which gives more detail on transparency for large donors. Transparency is likely to encourage scrutiny and lead to more effective behaviours, though it says little about the effectiveness of development finance more generally.
In addition to these measures, there are assessments being undertaken on the effectiveness of international organisations that receive aid. The Multilateral Organisation Performance Assessment Network provides institutional scores in four areas of organisational effectiveness but also covers “results” (development effectiveness).9 In addition, the Australian government (Australian Government & Australian AID 2012), the Ministry of Foreign Affairs of Denmark (2013), and the United Kingdom government (UK Government & Department for International Development 2011, 2013), among others, have undertaken and published their own reviews of multilateral organisations. Still, even if these reviews produce “ratings”, they largely reflect qualitative analysis. Furthermore, they are limited to multilateral organisations and are not available for countries own (“bilateral”) agencies. Still, to the extent that we know how much countries contribute to these organisations, we have some ways of assessing the “quality” of those contributions.
For OECD-DAC countries, and for the multilateral institutions that spend ODA, it is possible to calculate a number of aid effectiveness indicators using the OECD’s “Creditor Reporting System”. These indicators can relate to theory, evidence, or consensus about how they relate to quality. For these countries, the common reporting standard (CRS) provides a relatively comprehensive and comparable source of data at the project level to show where aid goes, the level of financial commitments and disbursements, what purposes it serves, and some descriptive information. However, it is up to the user of the data to conceptualise how these variables can be manipulated and analysed to produced aid quality indicators. Notably, many emerging development actors and providers of South-South cooperation do not report to the DAC, so CRS data is not available about them.
OECD-DAC countries also undertake systematic peer reviews, and these draw on quantitative measures as well as undertake qualitative assessments of development effectiveness. These are an important source of scrutiny, challenge, and mutual learning, and they often achieve high levels of engagement from ministers. Since the reviews follow a framework, it may be possible to systematically assess the findings across reviews—for example, on the elements of evaluation framework—and assign a quantitative score to the analysis.
So, for the 30 OECD-DAC countries and the group of around 13 countries that report to the DAC (OECD 2018a), relatively detailed data exists or can be feasibly created with publicly available data, but beyond these countries we have very limited quantitative information on the characteristics of their concessional development finance. Some admirable efforts have been made to use publicly available data to estimate the concessional element of development finance from Southern cooperation providers (United Nations Development Programme 2016) and also to identify six process and six outcome quality assessment guidelines. These are important efforts and, given the likely importance of transparency to effectiveness, Southern actors can surely accelerate progress towards the SDGs by providing common and consistent data on their concessional finance.
3.4 Quantifying Aid Quality
There have been relatively few attempts to quantitatively measure aid quality. Roodman (2013) developed a three-part aid quality measure that took a given quantity of aid and discounted it for tied aid, selectivity for less-poor and poorly governed recipient countries, and project proliferation. The resulting “quality-adjusted aid quantity” was used in the Commitment to Development Index through 2013. Easterly and Pfutze (2008) assessed and ranked 48 agencies quantitatively on aid “best practices” and included their own survey (with limited responses) regarding employment and administrative expenses to calculate overhead costs of agencies. Knack et al. (2010), in a World Bank policy research working paper, use a quantitative measure with 18 indicators using the Paris survey and OECD-DAC data. Birdsall and Kharas (2014) produced “QuODA”, the quality of official development assistance, from 2010 (based on data from 2008), which put together 30 indicators of aid quality and grouped them under four themes that aligned with the Paris principles of aid effectiveness. This enabled agencies to compare their “scores” in each of these four areas. Subsequently, Barder et al. (2016) combined QuODA scores for bilateral and multilateral donors to produce an “Aid Quality Index”, which was used in the Commitment to Development Index from 2014.
McKee and Mitchell (2018) produced an updated QuODA and were able to replicate or replace 24 indicators of aid effectiveness (see Annex A). Several of the original QuODA indicators were altered and replaced with GPEDC measures, which put a stronger emphasis on recipient views of aid (e.g. QuODA originally measured the effectiveness of recipients’ evaluation systems, but in the updated version using GPEDC measures, it checks whether evaluations are planned with recipients). As before, QuODA was fed into the Commitment to Development Index as the measure of “aid quality”, thereby giving even weight to each of the 24 indicators.
There are a number of significant limitations to these measures of aid quality including: heterogeneity in donor mandates and aid objectives whether there is any objective “optimum” allocation to aid recipients and whether fragmentation (allowing for greater competition and choice) or concentration (requiring less administrative burden) of donors is better for recipients (e.g. see Klingebiel et al. 2016). As noted above, there are limited direct links from measures of aid effectiveness to actual development impact. In the case of QuODA, a significant criticism is that it gives credit for aid going to poor and well-governed countries, but this runs counter to the need to give aid to fragile states, which are home to more than 60 per cent of the world’s extreme poor, and this figure expected to rise further (OECD 2018b, p. 99). Indeed, this challenge also applies to a number of indicators, including those collected by the GPEDC, which tends to emphasise alignment to, or use of, recipient country systems and frameworks that may be weaker in fragile states.
3.5 Concluding on Measures of Aid Effectiveness
In terms of measuring aid quality, there is a stated international consensus on the principles of effective aid from 2011, but little sign that these are the foundation of aid allocation, nor of recent or high-level interest from governments. There are some areas where evidence gives a clear view of some types of aid being more effective, including: avoiding tied aid, giving transparently and predictably, and giving to poorer countries. Still, in other areas, there is a lack of evidence, and tensions between different priorities. For example, there is a tension between giving in well-governed countries versus fragile states, and there may be a tension between using recipient systems and preventing leakage, particularly in fragile states. There may also be a tension between impacts that are easier to measure precisely, and programmes aiming for more systemic change.
There are few individual or aggregate measures of aid quality covering all actors. Nonetheless, there would still be value in producing and, crucially, comparing them. These comparisons would enable providers to see how their own development finance compared to others, prompt questions about differences, and ultimately improve approaches through learning. For DAC donors, much greater detail is possible, and it seems right that countries with high incomes have greater responsibility and should be held to a higher level of accountability.
4 Policies with Bilateral International Impacts
In considering what countries can do in terms of development cooperation, the contribution of development finance and aid is only one part of the overall picture. “Beyond aid” there are a number of other policies that have a major bearing on development. This is often referred to as the “policy coherence for development” agenda, which emphasises the role of different policy areas in affecting development. Policies and their coherence for development are being recognised in achieving the SDGs (Knoll 2014). The role of policy beyond aid in development has long been recognised. Since 2003, the Commitment to Development Index, developed by David Roodman (2005) and Nancy Birdsall (and now directed by me with colleagues of the Center for Global Development), has identified policy areas that affect development. It now tracks countries’ efforts in several policy areas, currently aid, environment, investment, migration, security, technology, and trade. Below, I expand on how policies matter in the flow of goods, people, finance, and ideas.
To illustrate the importance of policies beyond aid, consider China, which reduced the proportion of its population living on less than $1.90 per day from 66 to 0.7 per cent in only 25 years to 2015 (World Bank, n.d.-a), although it had benefitted from very little aid.10 Crucial elements of this transformation included domestic market reform and access to global markets, including membership in the World Trade Organization (Subramanian and Wei 2003).
Trade policy, then, is a crucial determinant in development. The substantial liberalisation of trade policy and the reduction in agricultural subsidies over the past four decades are likely to have played a key role in providing an international trade environment conducive to development. Of course, the recent trade disputes between the United States and China, along with the United Kingdom stepping back from its deep trade relationship with the European Union, illustrate that this environment is under threat, or even in reverse. As emerging economies grow, their markets and trade policies will be increasingly important to the development of others.
Migration is an area that has traditionally received less attention as a policy for development, often due to concerns with “brain drain”. My colleague Michael Clemens has refuted the idea that migration harms developing countries via a “brain drain” (Clemens 2009). He shows that 94 per cent of the countries that grew to middle-income status or higher between 1960 and 2013 also saw an increase in the proportion of their population emigrating (Clemens 2017, p. 5). Often the impacts on migrants themselves are forgotten in discussions of development and migration. Work by Clemens and Lant Pritchett shows that simply by moving from a poor country to a rich one, people can increase their incomes by 700–1000 per cent (Clemens 2011). This effect is a multiple of effective aid programmes (Pritchett 2016). The importance of migration is also illustrated through the sums of remittance flows to low- and middle-income countries: These amounted to $429 billion in 2016 (World Bank Group and Knomad 2018, p. 4)—almost four times the amount of ODA, and much more stable than private finance flows.
On ideas, countries’ policies centred on spreading ideas and knowledge also matter. One element of this effort if focussed on research and development, which we consider under GPGs. Other areas that matter are policies around knowledge transfer, for example on intellectual property rights (Park and Lippoldt 2014).
On finance, we have considered in the section above the quantity and quality of government-led development finance. But policy also governs several other important areas—in particular, policy on international tax plays a role in a developing country’s ability to raise tax revenue (domestic resource mobilisation). Countries also seek to influence or support private financial flows, for example with their investment policy and agreements. Work by the Research Centre for Policy Coherence and Development (CIECODE) (see Robinson et al. 2018, pp. 20–21) provides a framework for considering an equilibrium between ensuring that countries retain their right to regulate as they pursue public policy interests (including sustainable development objectives) while contributing to a favourable investment climate and protecting foreign investors from unjustified discrimination measures by the host state.
In summary, any assessment of countries’ development contributions should take these policy areas into account.
5 Global Public Goods
GPGs can be described simply as “goods, whose benefits or costs are of nearly universal reach or potentially affecting anyone anywhere” (Kaul 2013). In economics, examples of pure public goods are those where someone cannot be prevented from consuming a good (“non-excludable”) and where one person’s consumption does not detract from another’s (“non-rival”). Street lights have public good characteristics, and clean air is close to being a pure public good. GPGs are those public goods that are “global” or significantly transboundary. Examples include climate stability, security, and freedom from disease—we all benefit from these, but one country’s activity affects their overall supply.
A country’s contribution to GPGs is distinct from the policies in the prior section because one country’s activities affect all (or many) countries simultaneously, whereas in the prior section we were concerned with bilateral policy impacts. GPGs therefore require global—rather than bilateral—collaboration, and incentives are more diffuse.
In terms of countries’ contributions to GPGs, these can be positive or negative (and some activities—intentional or otherwise—create global public “bads” such as pollution). We can think of GPGs as assets that provide returns for a wide range of countries and people, but that countries can unilaterally add to, or deplete. The most obvious example is that of the climate. The Earth’s atmosphere is an asset that benefits the entire planet (to different degrees) by protecting us from the sun’s rays. Countries damage that asset when activity leads to greenhouse gas emissions, or they can contribute to that asset with forests, which absorb these gases and mitigate its depletion.
A key issue in accepting and allocating responsibility for maintaining GPGs is whether to consider the asset (i.e. the stock), or the current contributions to it (the flow). In this analysis, as we are concerned with current efforts, we focus on the latter, but it is clear that historic contributions are relevant in many cases, especially in assigning responsibility for enhancement or damage. Similarly, a country may contribute to a GPG by providing it directly, or by preserving it (which might include reducing or eliminating damaging activity).
Which GPGs should we be most concerned with? There is relatively little research assessing the value and importance of GPGs. Kenny et al. (2018) has produced a list and includes the categories of health (including vaccination); environment; economy; security; knowledge and technology; migration; and norms. I have attached a full list in Annex B. It is clear that countries’ actions in many of these areas have the potential to affect millions of lives overseas.
An important research question is the relative importance—and valuation—of these GPGs. There appears to be little work attempting to compare, say, the value of undertaking vaccination with climate mitigation. There have been calls for multilateral and bilateral development agencies to fund more projects that address these and other GPGs. As this would involve some trade-off of GPGs with other objectives—including poverty reduction—it is important for policy-makers to understand their relative value and effectiveness.
More effective development cooperation has the potential to lift large numbers of people out of extreme poverty. A common measurement between countries and institutions is extremely valuable in highlighting differences in approach, and it can proceed with or without targets, but it does rely on common standards and data. We can and should measure what we can now; if these comparisons are partial, this will signal clearly to policy-makers where better data is needed.
In this chapter, I have set out three main types of policy that matter for development, summarised in Table 11.4.
On development finance—including aid—there is a need for basic information on the quantity and terms of concessional finance. The Total Official Support for Sustainable Development measure is a helpful broader measure in development, but there remains a core question about how much concessional help is being funded by taxpayers, and it is important for recipients to understand whether any finance they receive is concessional or just effectively private lending through public bodies. New common standards and measures, which are not restricted to, or only led by, the OECD countries, are needed for all countries.
The political commitment to aid effectiveness and its measurement, at least in traditional OECD-DAC donors, has waned. The GPEDC, whose membership is broader, appears to lack buy-in from some key countries, although its data gathering is an important source of comparability in aid effectiveness, and it should be empowered and resourced to develop and extend its surveys. Countries will need to come together to agree how they will address effectiveness in the new environment where, for example, most of the extreme poor are potentially in fragile states. Without this focus on effectiveness, there is a risk that substantial sums of global aid are being used ineffectively, prolonging progress on the SDGs and keeping millions unnecessarily in poverty.
More generally, we need to be clear that concessional finance is not the only—or even perhaps the most important—policy supporting development. Policies around the flow of goods, people, and ideas are as important as those around capital. Similarly, GPGs receive too little attention, and little research exists that values these goods and considers the trade-offs between them and other areas. Common measures appear to be present in many of these issues.
The Center for Global Development will be continuing work in these areas, and it hopes to produce new measures of countries’ commitments to development, which apply to a broader group of major economies, and to evolve its measures of development finance. We hope these will support governments and leaders to reinvigorate their commitments to effectiveness, and help build a new consensus on effective aid and development that will accelerate the achievement of the global goals.
A fuller definition of “development” in this section would be “activities that directly enhance welfare of people—particularly those in poverty or extreme poverty—in another country; and/or that is likely to lead to stronger economic, environmental, or social growth”.This is a broad definition that, in principle, would encompass any policies or actions that affect other countries, including, for example, positions in international and regional fora. In practice, we are mainly concerned with those that have the biggest—or a material—impact on others. For example, a single vote at the UN or European Union would be unlikely to qualify as material, but failure to adopt certain international treaties would.
According to this measure, in 2017, several other countries would have qualified if the G20 was purely defined by GDP at market prices. For example, Nigeria, Iran, Poland, Switzerland, and Thailand have absolute GDPs higher than the smallest economy in the G20 (South Africa).
I explain the rationale for the groupings below. For a detailed description of the rationale and evidence supporting the impact of policies on development, see Robinson et al. (2018).
This suggestion was made by Pierre Jacquet in a December 2018 workshop at CGD on the future of measuring commitment to development.
The Global Partnership was created at the Fourth High-Level Forum on Aid Effectiveness in Busan in 2011 (GPEDC, n.d.-a).
For a current indicator framework, see the GPEDC (n.d.-b).
In 1968, the United Nations Conference on Trade and Development released a paper identifying and discussing the impact of tied aid. This report was followed by a condemnation of the practice by the Pearson Commission. Jepma (1991) found that the value of aid was reduced 13–23 per cent by the practice of tying. Despite recent progress, the OECD continues to push for untying aid (OECD, n.d.-d).
See the GPEDC (n.d.-b) for a current indicator framework.
“MOPAN assessments provide a snapshot of four dimensions of organisational effectiveness (strategic management, operational management, relationship management, and knowledge management), but also cover development effectiveness (results)” (Multilateral Organisation Performance Assessment Network, n.d.).
China received aid that grew from near zero in 1980 to a peak of 0.725 per cent of GDP in 1993 before falling to 0.146 per cent in 2000, and to 0.1 per cent in 2003 and below zero (i.e. a net donor) after 2010 (World Bank, n.d.-b).
AidData. (2018). The 2017 Listening to Leaders survey aggregate dataset. https://www.aiddata.org/data/the-2017-listening-to-leaders-survey-aggregate-dataset.
Australian Government & Australian AID. (2012). Australian multilateral assessment. https://dfat.gov.au/about-us/publications/Documents/ama-full-report-2.pdf.
Barder, O., Krylova, P., & Talbot, T. (2016). How much and how well: Revisiting the aid component of the Commitment to Development Index (CGD Policy Paper 085). Washington, DC: Center for Global Development.
Besharati, N. A., & Macfeely, S. (2019, March). Defining and quantifying South-South cooperation (UNCTAD Research Paper No. 30). Geneva: United Nations.
Birdsall, N., & Kharas, H. (2014). The quality of official development assistance (QuODA). 3rd ed. https://www.cgdev.org/sites/default/files/QUODA_final_revised_september.pdf.
Bracho, G. (2021). Failing to share the burden: Traditional donors, Southern providers, and the twilight of the GPEDC and the post-war aid system. In S. Chaturvedi et al. (Eds.), The Palgrave Handbook of Development Cooperation for Achieving the 2030 Agenda (pp. 367–391). Palgrave.
Clemens, M. A. (2009). Think again: Brain drain. https://www.cgdev.org/article/think-again-brain-drain-foreignpolicycom.
Clements, M. A. (2011). Trillion-dollar bills on the sidewalk: Why don’t more economists study emigration? https://www.cgdev.org/blog/trillion-dollar-bills-sidewalk-why-don%E2%80%99t-more-economists-study-emigration.
Clemens, M.A. (2017). Migration is a form of development: The need for innovation to regulate migration for mutual benefit (Population Division Technical Paper No. 2017/8). New York, NY: United Nations Department of Economic and Social Affairs.
Easterly, W., & Pfutze, T. (2008, Spring). Where does the money go? Best and worst practices in foreign aid. The Journal of Economic Perspectives, 22(2), 29–52.
Gavi, the Vaccine Alliance. (n.d.). Facts and figures. https://www.gavi.org/about/mission/facts-and-figures/.
Global Reporting Initiative. (2018). GRI Standards Download Center. https://www.globalreporting.org/standards/gri-standards-download-center/.
GPEDC (Global Partnership for Effective Development Co-operation). (n.d.-a). About the partnership. http://effectivecooperation.org/about/about-the-partnership/.
GPEDC. (n.d.-b). The monitoring framework of the global partnership. http://effectivecooperation.org/wp-content/uploads/2015/05/GPEDC-Monitoring-Framework-10-Indicators.pdf.
Howarth, C. N. (2017). Does development aid work? https://www.academia.edu/36050294/Does_Development_Aid_Work.
Jepma, C. J. (1991). The tying of aid. Paris: Organisation for Economic Co-operation and Development.
Kaul, I. (2013). Global public goods—A concept for framing the post-2015 agenda? (DIE Discussion Paper 2/2013). Bonn: German Development Institute / Deutsches Institut für Entwicklungspolitik (DIE).
Kenny, C., Snyder, M., & Patel, D. (2018). Measures of global public goods and international spillovers (CGD Working Paper 474). Washington, DC: Center for Global Development.
Klingebiel, S., Mahn, T., & Negre, M. (Eds.). (2016). The fragmentation of aid: Concepts, measurements and implications for development cooperation. London: Palgrave Macmillan.
Knack, S., Rogers, H. F., & Eubank, N. (2010). Aid quality and donor rankings. World Development, 39(11), 1907–1917. https://www.sciencedirect.com/science/article/abs/pii/S0305750X11002038.
Knoll, A. (2014, June). Bringing policy coherence for development into the post-2015 agenda (ECDPM Discussion Paper 163). Maastricht: European Centre for Development Policy Management.
Lankester, T. (2012). The politics and economics of Britain’s foreign aid: The Pergau Dam affair. London: Routledge.
McArthur, J. W., & Rasmussen, K. (2018). Who funds which multilateral organizations? https://www.brookings.edu/wp-content/uploads/2017/12/globalviews_who_funds_which_multilaterals.pdf.
McKee, C., & Mitchell, I. (2018). Quality of official development assistance: QuODA 2018 methodology. https://www.cgdev.org/sites/default/files/quoda-methodology-2018.pdf.
Meeks, P. (n.d.). Unravelling tied aid. https://eurodad.org/files/pdf/1546810-unravelling-tied-aid-1530880935.pdf.
Ministry of Foreign Affairs of Denmark. (2013). Danish multilateral development cooperation analysis. http://um.dk/da/~/media/UM/Danish-site/Documents/Danida/Samarbejde/Int-org/Danish%20Multilateral%20Development%20Cooperation%20Analysis.pdf.
Mitchell, I., Ritchie, E., & Rogerson, A. (2020). Finance for international development (CGD Working Paper 529). Washington, DC: Center for Global Development. https://www.cgdev.org/publication/finance-internationaldevelopment- fid.
Multilateral Organisation Performance Assessment Network. (n.d.). Mission statement. http://www.mopanonline.org/about/ourmission/.
Nunn, N., & Qian, N. (2014). U.S. food aid and civil conflict. American Economic Review, 104(6), 1630–1666.
OECD (Organisation for Economic Co-operation and Development). (2018a). Development co-operation report 2018: Joining forces to leave no one behind. Paris: OECD Publishing.
OECD. (2018b). States of fragility 2018. Paris: OECD Publishing.
OECD. (2019). Measuring distance to the SDG targets 2019: An assessment of where OECD countries stand. Paris: OECD Publishing.
OECD. (n.d.-a). The high level fora on aid effectiveness: A history. http://www.oecd.org/dac/effectiveness/thehighlevelforaonaideffectivenessahistory.htm.
OECD. (n.d.-b). The Busan Partnership for Effective Development Co-operation. http://www.oecd.org/development/effectiveness/busanpartnership.htm.
OECD. (n.d.-c). Aid at a glance charts. http://www.oecd.org/dac/financing-sustainable-development/development-finance-data/aid-at-a-glance.htm.
OECD. (n.d.-d). Untied aid. http://www.oecd.org/dac/financing-sustainable-development/development-finance-standards/untied-aid.htm.
OECD-DAC (Development Assistance Committee). (2018). Note on the treatment of loan concessionality in DAC statistics. http://www.oecd.org/dac/stats/concessionality-note.htm.
OECD-DAC. (n.d.). International TOSSD task force. http://www.oecd.org/dac/financing-sustainable-development/development-finance-standards/tossd-task-force.htm.
Park, W. G., & Lippoldt, D. C. (2014). Channels of technology transfer and intellectual property rights in developing countries. In S. Ahn, B. H. Hall, & K. Lee (Eds.), Intellectual property for economic development (pp. 33–89). Cheltenham, UK: Edward Elgar Publishing.
Pritchett, L. (2016, October 25). The least you can do for global poverty is better than the best you can do. https://www.cgdev.org/blog/least-you-can-do-global-poverty-better-best-you-can-do.
Publish What You Fund. (2018). Aid Transparency Index 2018. https://www.publishwhatyoufund.org/reports/2018-Aid-Transparency-Index.pdf.
Robinson, L., Käppeli, A., McKee, C., Mitchell, I., & Hillebrandt, H. (2018). The Commitment to Development Index: 2018 edition (Methodological Overview Paper). https://www.cgdev.org/sites/default/files/CDI-2018-methodology.pdf.
Roodman, D. (2005). The Commitment to Development Index: 2005 edition. https://www.cgdev.org/sites/default/files/archive/doc/cdi/technicaldescrip05.pdf.
Roodman, D. (2012). Due diligence: An impertinent inquiry into microfinance (CGD Brief). https://www.cgdev.org/sites/default/files/1425842_file_Roodman_Due_Diligence_brief_FINAL.pdf.
Roodman, D. (2013). The Commitment to Development Index: 2013 edition. https://www.cgdev.org/sites/default/files/archive/doc/CDI_2013/Index-technical-description-2013-final.pdf.
Scott, S. (2015). The accidental birth of “official development assistance” (OECD Development Co-operation Working Papers No. 24). Paris: OECD Publishing.
Subramanian, A., & Wei, S.-J. (2003, October). The WTO promotes trade, strongly but unevenly (NBER Working Paper No. 10024). Cambridge, MA: National Bureau of Economic Research.
UK Government & Department for International Development. (2011). Multilateral aid review 2011. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/67583/multilateral_aid_review.pdf.
UK Government & Department for International Development. (2013). Multilateral aid review update 2013. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/297523/MAR-review-dec13.pdf.
United Nations Development Programme. (2016). Concessional financial flows among southern countries: Conceptualising design principles, operational modalities and an assessment framework. https://www.undp.org/content/dam/undp/library/development-impact/SS%20Research%20Publications/11873%20-%20Concessional%20Financial%20Flows%20Among%20Southern%20Countries_Op%2008_Web%20Version(1).pdf.
World Bank. (2020). World Development Indicators online database. https://data.worldbank.org/data-catalog/world-development-indicators.
World Bank. (n.d.-a). Poverty headcount ratio at $1.90 a day (2011 PPP) (% of population). https://data.worldbank.org/indicator/SI.POV.DDAY?locations=CN.
World Bank. (n.d.-b). Net ODA received (% of GNI). https://data.worldbank.org/indicator/DT.ODA.ODAT.GN.ZS?locations=CN.
World Bank Group & Knomad. (2018). Migration and remittances: Recent developments and outlook. http://pubdocs.worldbank.org/en/992371492706371662/MigrationandDevelopmentBrief27.pdf.
I’m very grateful for excellent advice and input to this chapter from Arthur Baker, Caitlin McKee, Stephan Kyburz, Lee Robinson, and Owen Barder. All views and any errors are mine.
Editors and Affiliations
Annex A. Quality of official development assistance (QuODA) indicators
Below are the 24 indicators included in the 2018 update of QuODA. The last three columns identify where there is some support for the indicator. Full explanations of indicators and supporting sources are available in McKee and Mitchell (2018).
Inclusion supported by
Allocation to poor countries
Allocation to well-governed countries
High country programmable aid share
Focus/specialisation by recipient country
Focus/specialisation by sector
Support of select global public good facilities
Share of untied aid
Share of aid to recipients’ top development priorities
CRS; UN My World
Share of aid recorded in recipient budgets
Interventions using obj. from recipient frameworks
Use of recipient country systems
Share of scheduled aid recorded as received by recipients
Coverage of forward spending plans/aid predictability
Significance of aid relationships
Fragmentation across donor agencies
Median project size
Contribution to multilaterals
Transparency and learning
Membership in IATI
Info on development funding publicly accessible
Recording of project title and descriptions
Detail of project description
Reporting of aid delivery channel
Completeness of project-level commitment data
Share of evaluations planned with recipient
Annex B. List of global public goods
This annex provides a list of 39 measurable indicators of global public goods and six measures of other spillovers based on Kenny et al. (2018).
Antibiotics use in agriculture
WHO International Health Regulations Score—this measures surveillance, response, and preparedness for disease in a country
Vaccinations—benefits to home country but global spillovers
Exports of health-reducing products (such as tobacco)
Forests—a role in regulating environment. Forest area, protected areas such as national parks
Fish stocks agreements participation
Meat production—contributes to greenhouse gas emissions, deforestation, water use
Emissions of CO2, anthropogenic sulphur dioxide
Consumption of chlorofluorocarbons
Freshwater shocks and use
Membership in WTO—influences global rule-setting and upholds a rules-based system
Trade barriers—tariffs, subsidies, NTBs, STRI
ODA—in particular, funding for GPGs
Member of IMF—similar rationale as WTO
(negatively)—in receipt of IMF loan—due to risk of cross-border contagion
Contribution to UN budget
Good business practices—prevent a race to the bottom. Participation in:
The inclusive framework on base erosion and profit-shifting
Global forum on transparency and exchange of information for tax purposes
Convention on mutual administrative assistance in tax matters
Financial Secrecy Index
Crime and corruption—participation in:
UN Convention against Transnational Organized Crime
UN Convention against Corruption
Other economy spillovers:
Foreign direct investment
International Finance Corporation distance to frontier score—ease of doing business internationally
Cultivation of drugs that have major security and health spillovers globally—cannabis, cocaine (coca bush), opiates (opium poppy)
Consumption of these drugs
Exports of arms—particularly to countries with low respect for civil and political rights
Number of WMDs (e.g. nuclear) held
Contribution to UN peacekeeping budget
Participation in arms conventions—ATT, CCM, NPT, CTBT, CWC, BWC
Ratification of ICC’s Rome statute—ICC has brought cases of crimes against humanity
Global Cybersecurity Index value—a proxy for cybersecurity
Armed conflict-related deaths and terror-related deaths
Research and development expenditure
Total patent grants—patents encourage innovation
Scientific and technical journal articles
Internet access—in the form of internet exchange points—broader access facilitates greater global exchange of ideas and information
Refugees departing a country—scored negatively as a burden on the international system
Refugees entering a country
Other migration spillovers:
Visa restrictions—make movement and integration difficult
Remittances from a country
Participation in UN Convention on the Law of the Sea—the use of the sea and its resources
Basel convention on control of transboundary movements of hazardous wastes and disposal
Adoption of the metric system—to facilitate greater comparability across countries
International student ID card—to permit students from certain countries to access cultural institutions with student status in other countries
© 2021 The Author(s)
About this chapter
Cite this chapter
Mitchell, I. (2021). Measuring Development Cooperation and the Quality of Aid. In: Chaturvedi, S., et al. The Palgrave Handbook of Development Cooperation for Achieving the 2030 Agenda. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-57938-8_11
Publisher Name: Palgrave Macmillan, Cham
Print ISBN: 978-3-030-57937-1
Online ISBN: 978-3-030-57938-8