The measurement of corruption is an old challenge of both academics and the policy community, due to the absence of a unanimously agreed upon definition and the widespread (although inaccurate) belief that owing to its informal and hidden nature, corruption is an unobservable phenomenon. The articles in this issue challenge this belief. Individual corruption, as traditionally defined by Nye (1967) as private profit due to abuse of public authority is sometimes difficult to observe, especially in governance contexts where public integrity is the norm and corruption is the exception. In such environments, corrupt acts are often hidden and so are their proceeds: the corrupt people have to go to great length to hide their undue income in offshores or tax havens such as Panama or the Virgin Islands. The same is not true for governance contexts where corruption dominates, or, in another words, a particularistic distribution exists of public resources and goods ending in favorable treatment for some and discrimination for others, so the opposite of the modern administrative norm of ethical universalism (everyone treated equally and fairly). Lavish fortunes completely disproportionate with the lawful revenues of officials are displayed in front of everybody to see (since corrupt societies are, in Max Weber’s words, status societies, where wealth is only a vehicle to obtain greater status, so it has to be visible), votes are bought or stolen openly, the state’s assets are treated in a patrimonial way illegally or even legally (in corrupt countries, officials use government helicopters to go to weddings), as the law itself is particularistic and lawmakers first and foremost regulate legal exceptions in their own interest. Corrupt countries have their Virgin Islands downtown for everyone to observe and eventually measure.

The description above, anecdotic as it may sound, is hardly exceptional. About two thirds of the countries in the world are rated by the World Bank aggregate indicator Control of Corruption in the lower half of its score (1-5, see Fig. 1). Even if all experts are wrong (as this score is basically an aggregate of every expert opinion, also including some public opinion survey figures when available), sufficient grounds exist to argue that establishing corruption prevalence, especially to understand if it is the exception and we should deal with it at the individual level, or the norm (most government acts are particularistic) and we should approach it at the aggregate level is a worthwhile exercise (Kaufmann et al. 2006). The two governance contexts (equivalent to what North, Weingast and Wallis (2009) call open access order versus limited access order, Acemoglu and JA Robinson (2012) call inclusive versus extractive institutions and I call particularism versus ethical universalism (Mungiu-Pippidi 2006, 2015) require different approaches as to definition, mechanism, and measurement.

Fig. 1
figure 1

Distribution of frequency of World Bank Control of Corruption score. Legend: Original score recoded 1-10 with best performer Denmark 10. Source: World Governance Indicators, http://info.worldbank.org/governance/wgi/index.aspx#home

Table 1 illustrates how the two contexts differ, from definition to implications such as measurement or responses to corruption. Any individual is corrupt when engaging in a corrupt act, regardless of the side one is on (public or private). To define an organization or a country as corrupt we have to establish that corruption is the norm, in other words, that a majority (50 plus one) of their transactions (can be either defined as individual acts, or monetary value) are corrupt. In the first context, the corrupt agent is just a deviant and can be sanctioned by the principal if disclosed. In the second, the principal colludes with the agent, corruption being exercised through a pyramidal organization which extracts resources disproportionately in favor of the most powerful group, so anticorruption means solving power discretion and collective action problems. Apart from these two contexts, what remains to be distinguished is only corruption between private parties, which is best described as fraud or cheating if the corruption concept is to be bounded somewhere (as all good concepts should) instead of being stretched to cover all dishonest behavior involving money, even when no trespassing of public authority or trust exists.

Table 1 Corruption as governance context

In the development approach to corruption that we, as authors of this issue, endorse (see also Huther and Shah 2000; Mungiu-Pippidi 2006, 2015) corruption is thus defined as a practice deviating from the ethical universalism norm of public goods and resources distribution (as defined in law, rules, and the modern principles of administrative impersonality, impartiality and equality) resulting in undue benefit for individuals or groups (Parsons 1997; Mungiu-Pippidi 2006, 2015). This includes both illegal forms of undue profit and legal (Kaufmann and Vicente 2011), both monetary favors and other forms (nepotism, patronage, connections, any sort of particularism), both petty and grand (classifying corruption by size of bribes makes little sense anyway, unless context is defined, in corrupt contexts collusion exist across ‘grand’ and ‘petty’). Particularism results in privilege to some (favoritism) and discrimination to others, which can both be measured (Johnston 2013; Warren 2004). United Nations Convention against Corruption sets clear standards of ethical universalism described as impartiality, equal treatment, transparency and even participation. As over 170 countries have subscribed so far, these constitute the present legal as well as normative benchmarks against which we can measure corrupt practices in a country, a sector or an organization.

Nearly ten years have elapsed since their text, and due to big data development we are now able to present work which develop both the second and the third method further than to specific projects of limited coverage, allowing sub-national as well as cross-national comparisons. The universe of observations is given by all the transactions that a government agency, a sector or indeed a state engage in (from regulation to spending). The measurement efforts aim to understand how many of these transactions are as “they should be”, in other words impersonal, and how many are not (corruption prevalence), and they also combine 1, 2, and 3 analytically in order to observe two novel things, change over time in the capacity of a country to control corruption, in the presence or absence of specific anticorruption instruments. These results originate with the EU FP7 research program ANTICORRP (anticorrp.eu) and are presented in this issue papers by Fazekas et al. (procurement and companies ownership data), Doroftei (procurement and officials’ financial disclosures data), Vaz-Mondo (time series of official city hall audits in an electoral context), Vargas and Schlutz (financial disclosures in a macro context), and finally Mungiu-Pippidi and Dadasov (macro indicators enabling control of corruption). For comparison, this special issue also includes other recent attempts to use combined data, such as Picci and Escresa (number of trials on the basis of Foreign Corrupt Practices Act in an international trading context) or novel ways to interpret perception data (Rose, Dählstrom and Charron). Authors rely on a comparative method and use advanced inferential statistics and time series data in all ANTICORRP papers, combining the micro contract level analysis with the macro analysis at sub-national level (comparing across regions and municipalities) and national level (countries and continents). The data from the ANTICORRP project is therefore objective (procurement data, fiscal or e-govt data, financial disclosures, audits) and abundant, for the most part authors use big data and do not resort to sampling. The coverage is also universal, with sub-national papers focusing on Brazil and Romania, but the other papers having EU and global coverage. An important difference exists between papers which measure practices directly, either through public contract data, audits or questions on bribing in surveys, and those which estimate the capacity to constrain corruption indirectly, on the basis of a theoretical and statistical model. However, there is one major commonality: borders of national states or jurisdictions are essential to these measurements, either to compare across as in most papers (including the one on cross border bribing), or to use as control, as in the Brazil paper. In this respect, cross border crimes which blur boundaries, such as illegal financial flows, present comparative research with a limitation that an extension of national jurisdictions in the future (for instance, in the case of beneficial ownership) might help mitigate.

To conclude, most papers in this issue present important advances to the state of the art in corruption measurement. They are fact based, relying on objective and specific observable phenomena rather than perceptions; they are sensitive to change and policy intervention, therefore actionable upon. They allow far more precise diagnoses of governance and reform roadmaps at national and sub-national level than the previous generation of indicators based on experts’ perception (generation one) or surveys (generation two). This is not to discard the historical role of perception indicators, which first allowed for cross national comparisons—as some authors in this volume show, the ‘old’ perception indicators correlate to some extent to the new ones, although, of course, not on change (it would be difficult, since perception indicators were notorious for being ‘lagging’ and insensitive to change). More important still, the ‘new’ ones correlate across themselves (CRI based on procurement, IPI based on national integrity frameworks, and PACI based on litigation patterns weighted by trade all correlate), showing that governance practices are observable in clear patterns. The only limits to measurement are those of transparency, which governments still impose in many countries, because if governance practices became demonstrably corrupt most constituencies would demand substantial political changes.