Combining experimental evidence with machine learning to assess anti-corruption educational campaigns among Russian university students

Abstract

This paper examines how anti-corruption educational campaigns affect the attitudes of Russian university students toward corruption and academic integrity in the short run. About 2000 survey participants were randomly assigned to one of four different information materials (brochures or videos) about the negative consequences of corruption or to a control group. While we do not find important effects in the full sample, applying machine learning methods for detecting effect heterogeneity suggests that some subgroups of students might react to the same information differently, albeit statistical significance mostly vanishes when accounting for multiple hypotheses testing. Taking the point estimates at face value, students who commonly plagiarize appear to develop stronger negative attitudes toward corruption in the aftermath of our intervention. Unexpectedly, some information materials seem inducing more tolerant views on corruption among those who plagiarize less frequently and in the group of male students, while the effects on female students are generally close to zero. Therefore, policy makers aiming to implement anti-corruption education at a larger scale should scrutinize the possibility of (undesired) heterogeneous effects across student groups.

This is a preview of subscription content, access via your institution.

Fig. 1

Notes

  1. 1.

    Corruption can be defined as both “the abuse of entrusted power for private gain” (Transparency International) and “the lack of academic integrity”; see recent discussions with examples in Denisova-Schmidt (2017, 2019) and Denisova-Schmidt and de Wit (2017).

  2. 2.

    A large body of empirical literature suggests that women tend to be less corrupt; see Dimant and Tosato (2017), Dollar et al. (2001), Frank et al. (2011), Rivas (2013) and Swamy et al. (2001).

  3. 3.

    See, for instance, Romano and Wolf (2005, 2016), Lehrer et al. (2016), and Ludwig et al. (2017) for inference methods that account or correct for multiple hypothesis testing.

  4. 4.

    Alternatively, issues of multiple hypothesis testing could have been addressed by means of a pre-analysis plan outlining the methodology to be used for analyzing the data prior to collecting them, see for instance Nosek et al. (2018) for an in-depth discussion. A pre-analysis plan typically pre-specifies the hypotheses to be tested in a detailed way and includes a power calculation to determine the sample size required to detect effects with a specific probability. Such pre-registration prevents ex post snooping for statistically significant effects across outcomes or groups that have not been declared in the plan. Currie et al. (2019) demonstrate that even though discussions of pre-analysis plans are still rare in economics, they have been increasing sharply since 2012, after the American Economics Association decided to provide a registry for such plans. We acknowledge that no pre-registration was conducted for this study and for this reason, we apply specific machine learning methods to mitigate the issue of multiple hypothesis testing.

  5. 5.

    Here, “informal practices” refers to the practical norms that people often use in order to get things done.

  6. 6.

    Two sensitive questions about the students’ application of informal practices in their studies and students’ experience with bribery at the university were asked on a separate card and filled out by the interviewees themselves.

  7. 7.

    Reiderstvo, or asset-grabbing, is the illicit acquisition of a business or part of a business in Russia.

  8. 8.

    The General Assembly of the United Nations introduced Anti-Corruption Day in 2005 in order “to raise awareness of corruption and of the role of the Convention [against Corruption, resolution 58/4] in combating and preventing it” http://www.un.org/en/events/anticorruptionday/background.shtml.

  9. 9.

    The test statistic and the critical value are equal to 21.08 and 9.24, respectively.

  10. 10.

    The full list of covariates is given in Table A1 in Online Appendix A.

  11. 11.

    Based on the average of daily exchange rates from the Russian Central Bank in the period January 1 to November 1, 2016.

  12. 12.

    Primary and secondary education is predominantly public and tuition-free in Russia. However, informal payments at schools are widespread and range from covering basic maintenance of a school building and the provision of school guarding to some excessive school needs. While voluntary additional school payments have been ruled legal, the fees are often coercive in reality. Also, gift-giving to teachers can be voluntary or forced by parental committees or even the teachers themselves. Our data do not allow distinguishing between the two types in both the cases of additional school fees and gift-giving to teachers.

  13. 13.

    As pointed out by Lehrer and Xie (2018), minimizing the (unweighted) sum of squared residuals implicitly assumes homoskedastic errors.

  14. 14.

    To limit the complexity of trees, we apply cross-validation for determining the optimal number of splits. Furthermore, the minimum leaf (i.e., subgroup) size is set to 25 observations.

  15. 15.

    There are in total 124 treatment–outcome test combinations. Given our specification of recursive partitioning, splits are, however, not found for some combinations, which results in 95 primary-level splits.

  16. 16.

    Most of these differences across subgroups are statically significant at the 5% or 10% levels.

References

  1. Altbach PG (2016) Global perspectives on higher education. John Hopkins University Press, Baltimore

    Google Scholar 

  2. An W, Wan X (2016) R: local average response functions for instrumental variable estimation of treatment effects. https://cran.r-project.org/web/packages/LARF/. Accessed June 2017

  3. Armantier O, Boly A (2011) A controlled field experiment on corruption. Eur Econ Rev 55(8):1072–1082

    Article  Google Scholar 

  4. Armantier O, Boly A (2013) Comparing corruption in the laboratory and in the field in Burkina Faso and in Canada. Econ J 123(573):1168–1187

    Article  Google Scholar 

  5. Athey S, Imbens G (2016) Recursive partitioning for heterogeneous causal effects. Proc Natl Acad Sci 113(27):7353–7360

    Article  Google Scholar 

  6. Athey S, Imbens G, Kong Y, Ramachandra V (2016) An introduction to recursive partitioning for heterogeneous causal effects estimation using causalTree package. https://github.com/susanathey/causalTree. Accessed June 2017

  7. Athey S, Tibshirani J, Wager S (2019) Generalized random forests. Ann Stat 47:1148–1178

    Article  Google Scholar 

  8. Barr A, Serra D (2010) Corruption and culture: an experimental analysis. J Public Econ 94(11):862–869

    Article  Google Scholar 

  9. Belloni A, Chernozhukov V, Hansen C (2014) Inference on treatment effects after selection among high-dimensional controls. Rev Econ Stud 81(2):608–650

    Article  Google Scholar 

  10. Breiman L (2001) Random forests. Mach Learn 45:5–32

    Article  Google Scholar 

  11. Breiman L, Friedman J, Olshen R, Stone C (1984) Classification and regression trees. Wadsworth, Belmont

    Google Scholar 

  12. Cantoni D, Yang DY, Yuchtman N, Zhang YJ (2019) Protests as strategic games: experimental evidence from Hong Kong’s antiauthoritarian movement. Q J Econ 134(2):1021–1077

    Article  Google Scholar 

  13. Corbacho A, Gingerich DW, Oliveros V, Ruiz-Vega M (2016) Corruption as a self-fulfilling prophecy: evidence from a survey experiment in Costa Rica. Am J Polit Sci 60(4):1077–1092

    Article  Google Scholar 

  14. Currie J, Kleven H, Zwiers E (2019) Technology and big data are changing economics: mining text to track methods. Working paper, Princeton University

  15. Denisova-Schmidt E (2017) The challenges of academic integrity in higher education: current trends and outlook. CIHE perspectives, vol 5. Boston College, Boston

    Google Scholar 

  16. Denisova-Schmidt E (2019) Corruption in higher education. In: Teixeira N, Shin J-C (eds) Encyclopedia of international higher education systems and institutions. Springer, Berlin

    Google Scholar 

  17. Denisova-Schmidt E, de Wit H (2017) The global challenge of corruption in higher education. IAU Horizons 22(1):28–29

    Google Scholar 

  18. Denisova-Schmidt E, Huber M, Leontyeva E (2016) Do anti-corruption educational campaigns reach students? Some evidence from Russia and Ukraine. Educ Stud Mosc 1:61–83

    Google Scholar 

  19. Denisova-Schmidt E, Huber M, Prytula Y (2015) An experimental evaluation of an anti-corruption intervention among Ukrainian university students. Eurasian Geogr Econ 56(6):713–734

    Article  Google Scholar 

  20. Dimant E, Tosato G (2017) Causes and effects of corruption: what has past decade’s research taught us? A survey. J Econ Surv 32:335–356

    Article  Google Scholar 

  21. Dollar D, Fisman R, Gatti R (2001) Are women really the “fairer” sex? Corruption and women in government. J Econ Behav Organ 46(4):423–429

    Article  Google Scholar 

  22. Federal State Statistics Service (2016) Chislennost’ naselenia Rossiyskoy Federatsiy po municipalnim obrazovaniyam (The population of the Russian Federation by municipalities). Online bulletin, Federal State Statistics Service. http://www.gks.ru/wps/wcm/connect/rosstat_main/rosstat/ru/statistics/publications/catalog/afc8ea004d56a39ab251f2bafc3a6fce. Accessed June 2017

  23. Findley M, Nielson D, Sharman J (2014) Global shell games. Cambridge University Press, Cambridge

    Google Scholar 

  24. Frank B, Lambsdorff JG, Boehm F (2011) Gender and corruption: lessons from laboratory corruption experiments. Eur J Dev Res 23(1):59–71

    Article  Google Scholar 

  25. Holmes L (2015) Corruption: a very short introduction. Oxford University Press, Oxford

    Book  Google Scholar 

  26. Jetter M, Walker JK (2015) Good girl, bad boy: corrupt behavior in professional tennis. Working paper, Center for Research in Economics and Finance (CIEF)

  27. John LK, Loewenstein G, Rick SI (2014) Cheating more for less: upward social comparisons motivate the poorly compensated to cheat. Organ Behav Hum Decis Process 123(2):101–109

    Article  Google Scholar 

  28. Kasamara V, Sorokina A (2017) Rebuilt empire or new collapse? Geopolitical visions of Russian students. Eur Asia Stud 69(2):262–283

    Article  Google Scholar 

  29. Klemenčič M (2014) Student power in a global perspective and contemporary trends in student organising. Stud High Educ 39(3):396–411

    Article  Google Scholar 

  30. Knaus M, Lechner M, Strittmatter A (2017) Heterogeneous employment effects of job search programmes: a machine learning approach. CEPR Discussion Paper No. DP12224

  31. Knaus M, Lechner M, Strittmatter A (2018) Machine learning estimation of heterogeneous causal effects: empirical monte carlo evidence. Working paper, University of St. Gallen

  32. Korostelev A, Romenskiy V, Sagieva K. (Hosts) (2017) Progulka rasserzhennyh shkol’nikov: kak pokolenie YouTube vyshlo na ulicu i kak ego nakazhut (Angry pupils’ walk: how the YouTube generation went out into the streets and how it will be punished). In Pushkarev V, Yapparova L, Borzunova M, Alexandrov A, Zhelvnov A, Ruzavin P et al (eds) Zdes’ i sejchas. Vechernee shou [Here and now. The evening show]. Dozhd’, Moscow, Russia. https://tvrain.ru/teleshow/vechernee_shou/on_vam_ne_dimon-430761/. Accessed June 2017

  33. Lechner M (2019) Modified causal forests for estimating heterogeneous causal effects. CEPR Discussion Paper No. DP13430

  34. Lehrer SF, Pohl RV, Song K (2016) Targeting policies: multiple testing and distributional treatment effects. NBER Working Paper No. 22950

  35. Lehrer SF, Xie T (2018) The bigger picture: combining econometrics with analytics improve forecasts of movie success. NBER Working Paper No. 24755

  36. Ludwig J, Mullainathan S, Spiess J (2017) Machine learning tests for effects on multiple outcomes. Unpublished paper. https://scholar.harvard.edu/spiess/publications/machine-learning-tests-effects-multiple-outcomes

  37. Nosek BA, Ebersole CR, DeHaven AC, Mellor DT (2018) The preregistration revolution. Proc Natl Acad Sci 115:2600–2606

    Article  Google Scholar 

  38. Obrazovanie v Rossiiskoi Federatsii (2014) [Education in the Russian Federation: 2014] (2014). Statistical compilation, National Research Institute “Higher School of Economics”, Moscow, Russia

  39. Powers S, Qian J, Jung K, Schuler A, Shah NH, Hastie T, Tibshirani R (2018) Some methods for heterogeneous treatment effect estimation in high dimensions. Stat Med 37:1767–1787

    Article  Google Scholar 

  40. Rivas MF (2013) An experiment on corruption and gender. Bull Econ Res 65(1):10–42

    Article  Google Scholar 

  41. Romano JP, Wolf M (2005) Exact and approximate stepdown methods for multiple hypothesis testing. J Am Stat Assoc 100:94–108

    Article  Google Scholar 

  42. Romano JP, Wolf M (2016) Efficient computation of adjusted p-values for resampling-based stepdown multiple testing. Stat Probab Lett 113:38–40

    Article  Google Scholar 

  43. Serra D, Wantchekon L (2012) New advances in experimental research on corruption. Emerald, Bingley

    Book  Google Scholar 

  44. Spindler M, Chernozhukov V, Hansen C (2016) R: high-dimensional metrics. https://cran.r-project.org/web/packages/hdm/. Accessed June 2017

  45. Swamy A, Knack S, Lee Y, Azfar O (2001) Gender and corruption. J Dev Econ 64(1):25–55

    Article  Google Scholar 

  46. Tibshirani R (1996) Regresson shrinkage and selection via the lasso. J R Stat Soc 58:267–288

    Google Scholar 

  47. Transparency International Russia (2015a) Episode 1: Bribe. YouTube video. https://www.youtube.com/watch?v=zGeworhwEFo. Accessed July 2017

  48. Transparency International Russia (2015b) Episode 3: Corruption corporate raid. YouTube video. https://www.youtube.com/watch?v=4aTjUyX67xc. Accessed July 2017

  49. Volkov D (2017) Effekt ot filma “On vam ne Dimon” pochti proshel [The effect of the film “He is not Dimon to you” has almost passed]. Gazeta.ru. https://www.gazeta.ru/comments/2017/05/25_a_10691315.shtml. Accessed June 2017

  50. Wager S, Athey S (2018) Estimation and inference of heterogeneous treatment effects using random forests. J Am Stat Assoc 113:1228–1242

    Article  Google Scholar 

Download references

Acknowledgements

This study was conducted with financial support provided by the Center for Governance and Culture in Europe at the University of St. Gallen (HSG), Switzerland. The sponsor influenced neither the research design nor the interpretation of the results.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Elena Denisova-Schmidt.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The authors are thankful to participants of IX International Russian Higher Education Conference (RHEC), 2018, Moscow, for their feedback.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 571 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Denisova-Schmidt, E., Huber, M., Leontyeva, E. et al. Combining experimental evidence with machine learning to assess anti-corruption educational campaigns among Russian university students. Empir Econ 60, 1661–1684 (2021). https://doi.org/10.1007/s00181-020-01827-1

Download citation

Keywords

  • Anti-corruption campaigns
  • Experiments
  • Corruption
  • Academic integrity
  • University
  • Students
  • Russia

JEL Classification

  • D73
  • I23
  • C93