Methodological errors in corruption research: Recommendations for future research

Abstract

The secretive, illegal, multidimensional, and ubiquitous nature of corruption leads to formidable difficulties in research design and measurement. When research fails to account for these challenges, it can lead to an empirical misalignment with concepts and theories of corruption, with inferential errors commensurately emerging. We define, measure, and track four common measurement errors and two common research design errors for papers on corruption published in international business/management and political economy journals in the 2000–2021 period. Our data marks a substantial opportunity to tighten the fit between theory and methods. We offer recommendations to accelerate improvements in empirical research on corruption, and indeed for other phenomena that are characterized by legal, moral, and social desirability concerns. These empirical recommendations contribute to more robust theory building.

Résumé

La nature multidimensionnelle, omniprésente, illégale et secrète de la corruption entraîne des difficultés considérables en matière de conception et de mesure de la recherche. Lorsque la recherche ne parvient pas à tenir compte de ces difficultés, elle peut conduire à un décalage empirique avec les concepts et les théories de la corruption, ce qui mène proportionnellement à des erreurs d'inférence. Nous définissons, mesurons et suivons deux erreurs de conception de la recherche et quatre erreurs de mesure courantes pour les articles sur la corruption publiés dans les revues en Affaires Internationales/Management et Economie Politique au cours de la période 2000–2021. Nos données montrent qu'il est possible d'améliorer l'adéquation entre la théorie et les méthodes. Nous élaborons des recommandations pour accélérer les améliorations de la recherche empirique sur la corruption, ainsi que sur d'autres phénomènes caractérisés par des préoccupations d'ordre juridique, moral et de désirabilité sociale. Ces recommandations empiriques contribuent à une construction théorique plus robuste.

Resumen

La naturaleza hermética, ilegal, multidimensional y omnipresente de la corrupción lleva a dificultades enormes al diseñar investigaciones y medirla. Cuando los resultados de investigación no consiguen dan cuenta de estos desafíos, puede llevar a un desfase empírico con los conceptos y teorías de corrupción, con errores inferenciales proporcionalmente emergentes. Definimos, medimos y le dimos seguimiento a cuatro errores de medición comunes y dos diseños de investigación comunes en artículos científicos sobre corrupción publicados en revistas de Negocios Internacionales/Gerencia y Economía Política en el periodo 2000–2021. Nuestros datos señalan una oportunidad importante para ser más estrictos en el ajuste entre teoría y la metodología. Ofrecemos recomendaciones para acelerar los mejoramientos en la investigación empírica sobre corrupción, y de hecho para otros fenómenos que son caracterizados por preocupaciones legales, morales y de deseabilidad social. Estas recomendaciones empíricas contribuyen a una construcción teórica más robusta.

Resumo

A natureza secreta, ilegal, multidimensional e onipresente da corrupção leva a dificuldades formidáveis no planejamento e medição de pesquisas. Quando a pesquisa falha em dar conta desses desafios, ela pode levar a um desalinhamento empírico com conceitos e teorias de corrupção, com erros inferenciais surgindo proporcionalmente. Definimos, medimos e rastreamos quatro erros comuns de medição e dois comuns erros de design de pesquisa para artigos sobre corrupção publicados em revistas internacionais de negócios/gestão e economia política no período 2000–2021. Nossos dados marcam uma oportunidade substancial para aprimorar o ajuste entre teoria e métodos. Oferecemos recomendações para acelerar melhorias na pesquisa empírica sobre corrupção e, de fato, sobre outros fenômenos que são caracterizados por preocupações legais, morais e de interesse social. Essas recomendações empíricas contribuem para mais robusto desenvolvimento de teoria.

摘要

腐败的秘密、非法、多维和无处不在的性质导致在研究设计和测量方面的巨大困难。当研究未能解决这些挑战时， 可能会导致与腐败概念和理论的实证失调， 并相应地出现推论错误。我们定义、测量和跟踪了 2000–2021 年间在国际商业/管理和政治经济学期刊上发表的研究腐败的论文的四种常见的测量错误和两种常见的研究设计错误。我们的数据标志着加强理论与方法之间的匹配的重要机会。我们提出建议， 以加速改进对腐败的实证研究， 实际上是对其它以法律、道德和社会期望问题为特征的现象的研究。这些实证建议有助于建立更稳健的理论。

APPENDIX I: LIST EXPERIMENT IN MYANMAR

A list experiment is a technique that involves randomly assigning two versions of the same survey instrument to participants. This technique aids in lowering the perceived risk of detection by individual respondents, hence reducing the prevalence of social desirability error. The critical design element in a list experiment is a list of infrequent, yet non-sensitive activities, along with one other item. For the control group, this last item is a placebo, considered to have a near-zero probability of relevance in the local context. For the treatment group, this last item is a sensitive question, such as “paid informal charges to expedite application,” which measures the incidence of bribery. To shield respondents, they mark down how many activities they have engaged in but not which activities they engaged in. When aggregating anonymized responses to the group level, analysts can calculate the frequency of bribery by comparing the difference in means between groups. Subtracting the average number of behaviors in the treatment group from the same measure in the control group results in a direct estimate of the frequency of bribery within the sample.

Modifications to the list experiment can help scholars measure corruption and limit exposure to the six research methods errors. We use an example of a business survey in Myanmar to illustrate these modifications (Malesky et al., 2020). This list experiment aimed to reduce social desirability error, measure different types of corruption (petty and grand), and differentiate between the frequency and the scale of corruption. Curiously, Myanmar has traditionally scored poorly for corruption, but survey work has also found that petty corruption, as measured by direct questions about informal charges, was low and declining over time (Malesky et al., 2020). Although these findings point to a possible decline in corruption, this decline might be in petty but not in grand corruption. Exact conclusions cannot be drawn because grand and collusive corruption is more opaque than petty corruption. Further, grand corruption is subject to a pronounced social desirability error due to the potential for strong punishments.

To capture the scale of corruption, respondents must not only be shielded via a list experiment, but the experiment must be altered such that the respondents are asked to report on the total cost of a group of listed activities. The control group receives a list that includes a rare but plausible activity that is not corruption, and the treatment group receives the same list except that the rare but plausible activity is replaced by corruption. Respondents are asked to report the costs of listed activities in kyat (the local currency in Myanmar). Comparison of the treatment and control groups allows for the calculation of the average “price” of a bribe, which can also help untangle theoretical concerns about grand and petty corruption and their prevalence (Malesky et al., 2020).

The creative design element in this list experiment is the box that asks questions about the list. As not any one item from the list is ever revealed as having been done by a survey respondent, the respondents and their answers are appropriately shielded. We show a simplified version of the box from Malesky et al. (2020) in Figure A. This box is used to identify the scale of construction bribes. Respondents receive either Form A or Form B. The first three items in each form are non-sensitive, ordinary activities, randomly ordered, that are related to obtaining an operating license or a construction permit. However, Form A contains a sensitive activity related to bribery: “paid informal charge to expedite application.” Form B contains a placebo clause, such as “Paid for construction project simulation by digital-modeling company”. The placebo clause is plausible but unlikely, hence essentially taking a zero value for Form B respondents. As respondents are randomly distributed between Form A and Form B, any difference in aggregate values can be attributed to costs associate for informal charges paid to expedite applications.

See Table 5.

Table 5 Box for Myanmar list experiment to measure size of construction bribes

When analyzing the differences in aggregated responses, Form B respondents were found to have reported a mean of 404,335 kyat (USD 289) for all construction-related activities. The mean for firms answering Form A was 3,836,227 kyat (USD 2742) for all construction-related activities. These means are significantly different from one another, as the confidence intervals do not overlap. Subtracting the results of Form B from the results of Form A gives an average bribe size of 3.4 million kyat (USD 2453) per firm to obtain a construction permit. Taking this information together, this example illustrates that the techniques required are feasible, practical, and implementable. Moreover, the list experiment technique can not only reduce social desirability error, but it can also focus on a particular form of corruption, it can address questions related to frequency and scale, and it can yield information on actual experiences on corruption. Aside from the theory development benefits of this heightened measurement acuity, list experiments provide the potential to develop better data on corruption to track its incidence and evaluate intervention attempts to reduce either its frequency or scale.