Correcting for Scale Perception Bias in Measuring Corruption: an Application to Chile and Spain

Abstract

Corruption perceptions are commonly utilized as essential information to measure corruption across countries and regions. In this paper we consider the hypothesis that respondents to corruption perception questions utilize different response scales in their answers, i.e. for identical levels of corruption practices, subjects from one country could answer a different level of corruption perception than subjects from another country, based on socioeconomic characteristics. This hypothesis is investigated by utilizing the technique of anchoring vignettes with a sample of citizens in Spain and Chile. The results show that response scales are used differently across individuals in both countries, suggesting that the use of uncorrected measures of corruption perceptions could mislead some conclusions about the comparisons of the corruption levels between countries.

This is a preview of subscription content, access via your institution.

Fig. 1

Notes

  1. 1.

    See Mauro (1995), Tanzi and Davoodi (2001), Kauffman et al. (1999, 2000), Wei (1997), Ades and Di Tella (1997), Knack and Keefer (1995), LaPorta et al. (1999), Treisman (2000) and Idemudia et al. (2010). The World Bank has estimated that more than 109 million dollars are lost each year due to corruption, which represents 5 % of the global GDP. The African Union estimates that because of corruption, the African continent loses 25 % of GDP (Podobnik et al. 2008).

  2. 2.

    Olken (2009) argues that corruption imposes a tax on private production and public services.

  3. 3.

    Empirical studies have shown that corruption reduces GDP per capita (Mauro 1995; Lambsdorff 1999; Welsch 2008) and affects a wide range of economic indicators, such as public expenditure, total investment, capital inflows and foreign direct investment, international trade and foreign aid.

  4. 4.

    Recent applications of the estimation of corruption perception indexes can be found in Song and Cheng (2011), Rose-Ackerman (2004), Sah (2007), Connolly (2007), Welsch (2008), Anderson and Heywood (2009), Fan et al. (2009), Melgar et al. (2010), Di Tella and Schargrodsky (2003), Reinikka and Svensson (2004), Fisman and Wei (2004), Yang (2004), Hsieh and Moretti (2006) and Olken (2006). For example, Song and Cheng (2011) analyze corruption at local levels, Melgar et al. (2010) study individual characteristics and macroeconomic performance indicators that shape perceptions of corruption, Reinikka and Svensson (2004) examine corruption in educational expenditures, Fisman and Wei (2004) and Yang (2004) examine corruption in international trade, Di Tella and Schargrodsky (2003) examine corruption in hospital procurement, and Olken (2006) examines theft from a government redistribution program.

  5. 5.

    Transparency International combines assessments responses from asking (1) experts residing in the country and abroad, and (2) local business leaders. The World Bank governance indicators combine surveys of companies and individuals, the assessments of risk rating agency companies, NGO´s, and a number of multilateral aid agencies and other public sector organizations.

  6. 6.

    In a similar attempt to provide evidence of objectives data on corruption, Hsieh and Moretti (2006) compared prices received by Iraq under the U.N. Oil-for-Food program to the world oil price.

  7. 7.

    See King et al. (2004), Kapteyn et al. (2007), Salomon et al. (2004), Bago d’Uva et al. (2009), Kristensen and Johansson (2008b) and Agostini and Willington (2012).

  8. 8.

    The full questionnaire utilized for this study is available from the authors upon request.

  9. 9.

    The institutions are the African Development Bank, the Asian Development Bank, the Bertelsmann Foundation, the Economist Intelligence Unit, Freedom House, Global Insight, Political Risk Services, World Justice Project and the World Bank. It also utilizes four sources that reflect the views of business leaders about the countries where they reside: IMD, Political and Economic Risk Consultancy, Transparency International Bribe Payers Survey and the World Economic Forum. The World Bank governance indicators combine surveys of companies and individuals, the assessments of risk rating agency companies, NGO´s, and a number of multilateral aid agencies and other public sector organizations.

  10. 10.

    Results for the other domains of corruption perception analyzed in this study (tax efficacy and public employment) were similar to those presented in this paper and are available from the authors on request. Thus, throughout this paper we focus on modeling the domain of bribing practices of corruption utilizing both non-parametric and parametric approaches. This is actually the domain that can be considered that best represents corruption practices in both Spain and Chile.

  11. 11.

    The simulated levels of corruption perception are obtained by using HOPIT parameter estimations (eg first column of Table 5) and the average values of the corresponding explanatory covariates for each country (eg proportion of females, age, income, …). The HOPIT model presented in Table 5 includes dummy variables, which allow us to account for the possibility that the impact of explanatory covariates may be different among countries. That is, when calculating the perceived corruption level for each country, the bottom sections of Table 5 (eg Chile dummy interactions) are only employed for simulating Chile’s index (since the dummy takes value 0 for Spain).

  12. 12.

    Following King et al. (2004) recommendations, the few inconsistencies in the ordinal ranking found in the data set were treated as ties. The inefficiencies in this method come from the information lost due to this ties and ranking inconsistencies.

References

  1. Ades, A., & Di Tella, R. (1997). The new economics of corruption: A survey and some new results. Oxford: In Paul Heywood Political Corruption, Balckwell Pub.

    Google Scholar 

  2. Agostini, C. A., & Willington, M. (2012). El uso de Internet en Chile: evolución y factores determinantes. Persona y Sociedad, Universidad Alberto Hurtado, 24(1), 11–42.

    Google Scholar 

  3. Anderson, S., & Heywood, P. M. (2009). The politics of perception: Use and abuse of transparency international’s approach to measuring corruption. Political Studies, 57(4), 746–767.

    Article  Google Scholar 

  4. Araña, J.E., & León, C.J. (2005). Flexible mixture distribution modelling of dichotomous choice contingent valuation with heterogeneity. Journal of Environmental Economics and Management, 50, 170–188.

    Google Scholar 

  5. Araña, J.E., & León, C.J. (2012a). Scale perception bias in the valuation of environmental risks. Applied Economics, 44(20), 2607–2617.

    Google Scholar 

  6. Araña, J.E., & Leon, C.J. (2012b). The dynamics of preference elicitation after an environmental disaster: Stability and emotional load. Land Economics, 88(2), 362–381.

    Google Scholar 

  7. Bago d’Uva T., Lindeboom M., O’Donnell O., & Van Doorslaer E. (2009). Slipping anchor? Testing the vignettes approach to identification and correction of reporting heterogeneity. Tinbergen Institute Discussion Paper No. 09–091/3.

  8. Connolly, M. (2007). Measuring the effect of corruption on sovereign bond ratings. Journal of Economic Policy Reform, 4, 309–323.

    Article  Google Scholar 

  9. Di Tella R., & Schargrodsky E. (2003). The role of wages and auditing during a crackdown on corruption in the city of Buenos Aires. Journal of Law and Economics, 46.

  10. Donchev D., & Ujhelyi G. (2008). What do corruption indices measure?. University of Houston Economics Department, working paper (version of May 23, 2008).

  11. Effron, B., & Tibshirani, R. (1986). Bootstrap methods for standard errors confidence intervals and other measures of statistical. Statistical Science, 1(1), 54–75.

    Article  Google Scholar 

  12. Fan, C. S., Lin, C., & Treisman, D. (2009). Political decentralization and corruption: Evidence from around the world. Journal of Public Economics, 93, 14–34.

    Article  Google Scholar 

  13. Fisman, R., & Wei, S.-J. (2004). Tax rates and tax evasion: Evidence from ‘Missing Imports’ in China. Journal of Political Economy, 112(2), 471–500.

    Article  Google Scholar 

  14. Geweke, J., Keane, M. (2007). Smoothly mixing regressions. Journal of Econometrics, 138(1), 252–290.

    Google Scholar 

  15. Hopkins, D., & King, G. (2010). A method of automated nonparametric content analysis for social science. American Journal of Political Science, 54(1), 229–247.

    Article  Google Scholar 

  16. Hsieh, C.-T., & Moretti, E. (2006). Did Iraq cheat the United Nations? Underpricing, bribes, and the oil for food program. Quarterly Journal of Economics, 121, 1211–1248.

    Google Scholar 

  17. Idemudia, U., Cragg, W., & Best, B. (2010). The challenges and opportunities of implementing the integrity pact as a strategy for combating corruption in Nigeria’s oil rich Niger Delta Region. Public Administration and Development, 30, 227–290.

    Article  Google Scholar 

  18. INE (2011). Encuesta de Equipamiento y Uso de Tecnologías de Información y Comunicación en los Hogares.

  19. Kapteyn, A., Smith, J. P., & Van Soest, A. (2007a). Vignettes and self reports of work disability in the United States and the Netherlands. American Economic Review, 97(1), 461–473.

    Article  Google Scholar 

  20. Kapteyn A., Smith J. P., Van Soest A., Vonkova H. (2007). Anchoring vignettes and response consistency. RAND Working Paper Series WR-840.

  21. Kauffman D., Kraay A., Zoido-Lobaton P. (1999). Governance matters. Policy Research Working paper, 2196, World Bank.

  22. Kauffman D., Kraay A., Zoido-Lobaton P. (2000). Governance matters: From measurement to action. Finance and Development, International Monetary Fund.

  23. Kaufmann D., Kraay A., & Mastruzzi M. (2009). Governance matters VIII: Aggregate and individual governance indicators, 1996–2008. World Bank, Working Paper.

  24. King, G., Murray, C. J. L., Salomon, J. A., & Tandon, A. (2004). Enhancing the validity of cross-cultural comparability of measurement in survey research. American Journal of Political Science Review, 98(1), 191–207.

    Google Scholar 

  25. King, G., & Wand, J. (2007). Comparing incomparable survey responses: New tools for anchoring vignettes. Political Analysis, 15, 46–66.

    Article  Google Scholar 

  26. Knack, S., & Keefer, P. (1995). Institutions and economic performance: Crosscountry tests using alternative institutional measures. Economics and Politics, 7(3), 207–227.

    Article  Google Scholar 

  27. Kristensen, N., & Johansson, E. (2008). New evidence on cross-country differences in job satisfaction using anchoring vignettes. Labour Economics, 15, 96–117.

    Article  Google Scholar 

  28. Lambsdorff J.G. (1999). Corruption in empirical research. Transparency International, Working Paper.

  29. Lambsdorff J.G. (2003). Background Paper to the 2003 Corruption Perceptions Index. Transparency International, Working Paper.

  30. LaPorta, R., Lopez-de-Silanes, F., Shleifer, A., & Vishny, R. (1999). The quality of government. Journal of Law, Economics, and Organizations, 15(1), 222–279.

    Article  Google Scholar 

  31. Leon, C.J., Araña, J.E., & de Leon, J. (Forthcoming). Valuing the impact of corruption on subjective well being (SWB) using the technique of vignettes. Applied Economics.

  32. Mauro, P. (1995). Corruption and growth. Quarterly Journal of Economics, 110(3), 681–712.

    Article  Google Scholar 

  33. Melgar, N., Rossi, M., & Smith, T. W. (2010). The perception of corruption. International Journal of Public Opinion Research, 22(1), 120–131.

    Article  Google Scholar 

  34. Mocan, N. (2008). What determines corruption. International Evidence from Microdata. Economic Inquiry, 46(4), 493–510.

    Article  Google Scholar 

  35. Murray, C. J. L., Ozaltin, E., Tandon, A., Salomon, J., Sadana, R., & Chatterji, S. (2003). Empirical evaluation of the anchoring vignettes approach in health surveys. In C. J. L. Murray & E. DB (Eds.), ) vans Health systems performance assessment: debates, methods and empiricism (pp. 369–399). World Health Organisation: Geneva.

    Google Scholar 

  36. Olken, B. (2006). Corruption and the cost of redistribution. Journal of Public Economics, 90(4–5), 853–870.

    Article  Google Scholar 

  37. Olken, B. (2007). Monitoring corruption: Evidence form a field experiment in Indonesia. Journal of Political Economy, 115(2), 200–249.

    Article  Google Scholar 

  38. Olken, B. (2009). Corruption perceptions vs. corruption reality. Journal of Public Economics, 93, 950–964.

    Article  Google Scholar 

  39. Peracchi F., Rossetti C. (2010). The heterogeneous thresholds ordered response model: Identification and inference. EIEF Working Paper 10/12.

  40. Podobnik, B., Shao, J., Njavro, D., Plamen, I., & Stanley, H. E. (2008). Influence of corruption on economic growth rate and foreign investment. The European Physical Journal B, 63, 547–550.

    Article  Google Scholar 

  41. Razafindrakoto, M., & Roubaud, F. (2005). How far can we trust the experts’ opinion on corruption? An experiment based on surveys in francophone Africa. In transparency international, Global corruption report 2005. (pp. 292–295). London/Ann Arbor: Pluto Press.

    Google Scholar 

  42. Razafindrakoto, M., & Roubaud, F. (2010). Are international databases on corruption reliable? A comparison of expert opinion surveys and household survey in sub-saharan Africa. World Development, 38(8), 1057–1069.

    Article  Google Scholar 

  43. Reinikka, R., & Svensson, J. (2004). Local capture: Evidence from a central government transfer program in Uganda. Quarterly Journal of Economics, 119(2), 679–706.

    Article  Google Scholar 

  44. Rice, N., Robone, S., & Smith, P. (2010). Analysis of the validity of the vignette approach to correct for heterogeneity in reporting health system responsiveness. European Journal of Health Economics, 12(2), 141–162.

    Article  Google Scholar 

  45. Rose-Ackerman S. (2004). The challenge of poor governance and corruption. Paper presented at the Copenhagen Consensus Challenge and Yale University.

  46. Sah, R. (2007). Corruption across countries and regions: Some consequences of local osmosis. Journal of Economic Dynamics and Control, 31, 2573–2598.

    Article  Google Scholar 

  47. Salomon, J. A., Tandon, A., & Murray, C. J. L. (2004). Comparability of self rated health: Cross sectional multi-country survey using anchoring vignettes. British Medical Journal, 328(7434), 258–260.

    Article  Google Scholar 

  48. Seligson, M. (2006). The measurement and impact of corruption victimization: Survey evidence from Latin America. World Development, 34(2), 381–404.

    Article  Google Scholar 

  49. Song, X., & Cheng, W. (2011). Perception of corruption in 36 major Chinese cities: Based on survey of 1,642 experts. Social Indicators Research,. doi:10.1007/s11205-011-9896-4,1-11.

    Google Scholar 

  50. Tanzi, V., & Davoodi, H. (2001). Corrupción, inversión pública y crecimiento. Gestión y Análisis de Políticas Públicas, 21, 73–82.

    Google Scholar 

  51. Treisman, D. (2000). The causes of corruption: a cross-national study. Journal of Public Economics, 76, 399–457.

    Article  Google Scholar 

  52. Van Soest A., Delaney L., Harmon C., Kapteyn A., Smith J.P. (2007). Validating the use of vignettes for subjective threshold scales. IZA discussion paper 2860, Institute for the Study of Labor, Bonn.

  53. Wei S.-J. (1997). How Taxing is Corruption on International Investors. Working paper, 6030, National Bureau or Economic Research, Cambridge, Mass.

  54. Welsch, H. (2008). The welfare costs of corruption. Applied Economics, 40, 1839–1849.

    Article  Google Scholar 

  55. Yang D. (2004). Can Enforcement Backfire? Crime displacement in the context of a common customs reform. Working paper, University of Michigan.

Download references

Acknowledgments

The authors gratefully acknowledge financial support by the Spanish Ministry of Science and Innovation (Special Programme for R&D projects codes ECO2008-06148, ECO2009-12629 and ECO2011-30365), and by the Agencia Canaria de Investigacion (Projects SolSubC 200801000381 and PI2007/040). Suggestions by the editor and three anonymous reviewers of this journal also helped to shape the piece. Seminar participants at Center for the Study of Choice (CenSoc), University of Technology, Sydney (February, 2011) and at the Waikato Management School, University of Waikato (June 2011) also providing remarks that improved the manuscript.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Javier de León.

Appendices

Appendix 1: The Vignette Equivalence Assumption

The methodology of anchoring vignettes relies on the assumption that all survey respondents interpret the levels of the categories described in the vignettes in the same way, i.e. the vignette equivalence assumption. Following Rice et al. (2010), a minimal condition for vignette equivalence is that the individual responses are consistent with the global ordering of the vignettes. Thus, for each domain and for each country we evaluate the proportion of respondents that gave an ordering of vignettes consistent with the global ordering. The latter can be derived by pooling all vignette responses across subsamples and considering the average categorical response for each vignette.

Table 7 presents the results of the proportions of consistent orderings by vignette and country. For each domain and for each vignette there was no substantial variation across both countries. The values of the percentages are all above 0.72, with a maximum rate of 0.98. On average, the percentages of consistent orderings are slightly larger for the domain of tax effectiveness than for the other two domains of corruption perception (public employment and bribing practices). Thus, these results give support to the maintained assumption of vignette equivalence.

Table 7 Proportions of consistent orderings by vignettes and country

Appendix 2: The Non-parametric Model

In order to explore whether or not data from different countries present scale perception bias, we follow King et al. (2004) recommendations and apply a non-parametric approach as a first step in the data analysis strategy. The main idea of this model is to “recode the categorical self-assessment relative to the set of vignettes” (King et al. 2004). If all the respondents order the vignettes in a similar way, individual responses can be recoded as a number that indicates the position of the self-assessment compared with the proposed vignettes.

More concisely, if V1, V2, V3 y V4 are the four vignettes proposed, and all the respondents rate V4 higher than V3, V3 higher than V2 and V2 higher than V1, then the individual response can be recoded in the following way.Footnote 12 The recoded variable would take a value of 1, if self-assessment value is lower than V1; a value of 2, if self-assessment value is the same than V1; a value of 3, if self-assessment value is higher than V1 but lower than V2; a value of 4, if self-assessment value is the same than V2; a value of 5, if self-assessment value is higher than V2 but lower than V3; a value of 6, if self-assessment value is the same than V3; a value of 7, if self-assessment value is higher than V3 but lower than V4; a value of 8, if self-assessment value is the same than V4; and a value of 9, if self-assessment value is higher than V4. That is, the original self-assessment variable for individual i (y i ), can be recoded as the variable C i that can be interpreted as a DIF-free variable. The recoded C i variable takes the following values:

$$ C_{i} = \left\{ {\begin{array}{*{20}c} 1 \hfill & {{\text{if}}\,y_{i} < V1} \hfill \\ 2 \hfill & {{\text{if}}\,y_{i} < V1} \hfill \\ 3 \hfill & {{\text{if}}\,V1 < y_{i} < V2} \hfill \\ 4 \hfill & {{\text{if}}\,y_{i} < V2} \hfill \\ 5 \hfill & {{\text{if}}\,V2 < y_{i} < V3} \hfill \\ 6 \hfill & {{\text{if}}\,y_{i} < V3} \hfill \\ 7 \hfill & {{\text{if}}\,V3 < y_{i} < V4} \hfill \\ 8 \hfill & {{\text{if}}\,y_{i} < V4} \hfill \\ 9 \hfill & {{\text{if}}\,y_{i} < V2} \hfill \\ \end{array} } \right. $$

Appendix 3: The Hopit Model

The model for the correction of the scale response bias using anchoring vignettes is based on a simultaneous equation approach that jointly models the self-assessment response to the corruption perception question and the responses to the vignette questions. The model is a generalization of the ordinal probit model, also called HOPIT model (e.g. King et al. 2004; King and Wand 2007; Kapteyn et al. 2007; Rice et al. 2010) that allows for the thresholds defined by categorical variables to be modeled as a function of covariates. Let c rj be the categorical self-assessment response of corruption perception framed into question r by individual j that can take a value from 1 to 5. Thus, we have the following behavioral equation:

$$ \tilde{c}_{rj} = \Upgamma z_{rj} + \zeta_{rj} $$
(1)

where Г is a vector of parameters, z rj is a vector of socioeconomic covariates that explains the self-assessment responses, and \( \zeta_{rj} \sim {\text{N (0, }}\sigma_{r}^{2} ) \). Variable \( \tilde{c}_{rj} \) is a latent or unobserved variable for which the researcher only observes the categorical response c rj which is assumed to be related with \( \tilde{c}_{rj} \)in the following way:

$$ c_{rj} = j\;{\text{if}}\;\tau_{j}^{i - 1} \le \tilde{c}_{rj} < \tau_{j}^{i} ,\quad i = 1, \ldots ,5 $$
(2)

where τ i j are the thresholds between categories that are given by:

$$ \tau_{j}^{0} = - \infty ,\quad \tau_{j}^{5} = \infty ,\quad \tau_{j}^{i} = \beta^{i} x_{j} + \mu_{j} ,\quad i = 1, \ldots ,4 $$
(3)

where the reporting behavior equation in (3) for τ i j models the thresholds as a function of socioeconomic covariates x j and parameter vector β i The error terms μ j are assumed to be normally distributed with zero mean and σ μ variance, and introduces the unobserved individual effect or heterogeneity in the response scale. In addition, μ j is assumed to be independent of ζ rj and z rj . Further, the error term in the threshold or reporting behavior Eq. (3) implies that the evaluations of corruption in the vignettes are correlated with each other and with the self-assessment of corruption, because some respondents might use low thresholds while others might use high thresholds.

The hypothesis of differential item functioning (DIF) follows because of the fact that respondents use different response scales, i.e. this means that τ i j are different across respondents because they depend on socioeconomic characteristics or covariates. The restricted model that does not allow for DIF does not allow variation of the thresholds across respondents. The unrestricted model estimates the reporting behavior equation in (3) simultaneously by utilizing the responses to the vignette questions.

That is, let c vj be the categorical response of corruption perception to the vignette questions by individual j that can take a value from 1 to 5. We assume that the expected value of the underlying latent variable \( \tilde{c}_{vj} \) depends only on the corresponding vignette, i.e.

$$ \tilde{c}_{vj} = \alpha_{v} + \varepsilon_{vj} $$
(4)

where α v is a fixed vignette parameter that represents the level of corruption represented in the vignette question v, and \( \varepsilon_{vj} \sim {\text{N (0, }}\sigma_{v}^{2} ) \).

Similarly to the self-assessment response model outlined in Eqs. (13), variable \( \tilde{c}_{vj} \) is a latent or unobserved variable for which the researcher only observes the categorical response for each vignette c vj which is assumed to be related with the underlying \( \tilde{c}_{vj} \)in the following way:

$$ c_{vj} = j\;{\text{if}}\tau_{j}^{i - 1} \le \tilde{c}_{vj} < \tau_{j}^{i} ,\quad i = 1, \ldots ,5 $$
(5)

where τ i j are the threshold between categories that are given by:

$$ \tau_{j}^{0} = - \infty ,\quad \tau_{j}^{5} = \infty ,\quad \tau_{j}^{i} = \beta^{i} x_{j} + \mu_{j} ,\quad i = 1, \ldots ,4 $$
(6)

where it can be seen that Eq. (6) is identical to Eq. (3). Therefore, the information provided by the individuals’ assessments to the levels of corruption defined by the anchoring vignettes allows us to correct for DIF by estimating the thresholds as functions of relevant socioeconomic variables simultaneously with the self-assessment responses.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

León, C.J., Araña, J.E. & de León, J. Correcting for Scale Perception Bias in Measuring Corruption: an Application to Chile and Spain. Soc Indic Res 114, 977–995 (2013). https://doi.org/10.1007/s11205-012-0185-7

Download citation

Keywords

  • Anchoring vignettes
  • Corruption perception index
  • Corruption perceptions
  • Transparency international