Skip to main content
SpringerLink
Log in

Overcoming Political Resistance to Evaluating the Environmental Impacts of EU Cohesion Policy

Überwindung des politischen Widerstands gegen Evaluationen von Umweltauswirkungen der EU-Kohäsionspolitik

  • Critical paper
  • Published:
Politische Vierteljahresschrift Aims and scope Submit manuscript

Abstract

Environmental pollution diminishes our quality of life. Counterfactual impact evaluations could help improve the implementation and effectiveness of environmental programs financed by European Union cohesion policy. Yet despite the possible benefits they might bring, such evaluations of these programs have been scarce. In this study on the Czech Republic, we identify both supportive political aspects and reasons for the low political acceptance of evaluations in the field of environmental pollution. Our study reveals several significant obstacles hindering the support and acceptance of evaluation results. First, there is a lack of data, and of what data there is, much of the existing panel data is of poor quality. Second, different public administration bodies often accept only their own data and are not ready to cooperate and share data. Finally, environment is a cross-border issue requiring cooperation among different national public administrations. In conclusion, political strategies are needed in order to take advantage of counterfactual impact evaluations and existing administrative data in the public sector.

Zusammenfassung

Umweltverschmutzung vermindert die Lebensqualität. Eine kontrafaktische Wirkungsanalyse kann dazu beitragen, die Durchführung und Wirksamkeit der aus der EU-Kohäsionspolitik finanzierten Umweltprogramme zu verbessern. Doch trotz dieses möglichen Nutzens sind Evaluationen solcher Programme selten. Der aktuelle Beitrag identifiziert begünstigende politische Faktoren, aber auch Gründe für die geringe politische Akzeptanz von kontrafaktischen Wirkungsanalysen in der Umweltpolitik der Tschechischen Republik. Er analysiert mehrere signifikante Hindernisse, die die Unterstützung und Akzeptanz von Evaluationsergebnissen behindern. Zu diesen zählen zunächst der allgemeine Datenmangel und qualitativ minderwertige Paneldaten. Zweitens akzeptieren diverse öffentliche Verwaltungen nur eigene Daten und sind zu einem Datenaustausch nicht bereit oder nicht in der Lage. Drittens ist Umweltverschmutzung ein ressortüberschreitendes Thema, das Zusammenarbeit zwischen den verschiedenen nationalen öffentlichen Verwaltungen erfordert. Aus diesen Gründen sind neue politische Strategien erforderlich, um die Vorteile von kontrafaktischen Wirkungsanalysen und die Existenz von Verwaltungsdaten im öffentlichen Sektor effektiv zu nutzen.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Counterfactual impact evaluations apply statistical methods to estimate the actual effects of policies; see Sect. 2 for more information concerning the pros and cons of these types of evaluations.

References

  • Abadie, Alberto, and Javier Gardeazabal. 2003. The economic costs of conflict: a case study of the Basque Country. American Economic Review 93(1):113–132. https://doi.org/10.1257/000282803321455188.

    Article  Google Scholar 

  • Alecke, Björn, Timo Mitze, Janina Reinkowski, and Gerhard Untiedt. 2012. Does firm size make a difference? Analysing the effectiveness of R&D subsidies in east Germany. German Economic Review 13(2):174–195. https://doi.org/10.1111/j.1468-0475.2011.00546.x.

    Article  Google Scholar 

  • Alkin, Marvin C., and Jean A. King. 2016. The historical development of evaluation use. American Journal of Evaluation 37(4):568–579. https://doi.org/10.1177/1098214016665164.

    Article  Google Scholar 

  • ASVAPP. 2012. Counterfactual impact of cohesion policy: impact and cost-effectiveness of investment subsidies in Italy. Final Report to DG Regional Policy.

    Google Scholar 

  • Batterbury, Sarah C. E. 2006. Principles and purposes of European Union Cohesion Policy evaluation. Regional Studies 40(2):179–188. https://doi.org/10.1080/00343400600600504.

    Article  Google Scholar 

  • Becker, Sascha O., Peter H. Egger, and Maximilian von Ehrlich. 2012. Too much of a good thing? On the growth effects of the EU’s regional policy. European Economic Review 56(4):648–668. https://doi.org/10.1016/j.euroecorev.2012.03.001.

    Article  Google Scholar 

  • Belfiore, Eleonora, and Oliver Bennett. 2007. Rethinking the social impacts of the arts. International Journal of Cultural Policy 13(2):135–151. https://doi.org/10.1080/10286630701342741.

    Article  Google Scholar 

  • Bienias, Stanisław, Tomasz Gapski, Jakub Jąkalski, Iwona Lewandowska, Monika Mackiewicz, Elżbieta Opałka, and Piotr Strzęboszewski. 2009. Evaluation in Poland: brief overview of evaluation process of EU Cohesion Policy funds. In Evaluation systems in the Visegrad member states, ed. Stanisław Bienias, Iwona Lewandowska. Warsaw: Ministry of Regional Development.

    Google Scholar 

  • Blažek, Zdeněk, Libor Černikovský, Ewa Krajny, Blanka Krejčí, Leszek Ośródka, Vladimíra Volná, and Marek Wojtylak. 2013. Vliv meteorologických podmínek na kvalitu ovzduší v přeshraniční oblasti Slezska a Moravy. Praha: Český hydrometeorologický ústav.

    Google Scholar 

  • Bruyninckx, Hans. 2009. Environmental evaluation practices and the issue of scale. New Directions for Evaluation 2009(122):31–39. https://doi.org/10.1002/ev.293.

    Article  Google Scholar 

  • Brůha, Jan, and Oto Potluka. 2015. Evaluace dopadů veřejných výdajových programů na kvalitu ovzduší. Evaluační teorie a praxe 3(2):61–78.

    Google Scholar 

  • Čadil, Jan, Karel Mirošník, and Ludmila Petkovová. 2016. Impact of R&D subsidies on enterprise performance in the Czech Republic. Society and Economy 38(3):387–398. https://doi.org/10.1556/204.2016.38.3.7.

    Article  Google Scholar 

  • CENIA. 2013. Zpráva o životním prostředí České republiky

    Google Scholar 

  • Černikovský, Libor, Blanka Krejčí, Zdeněk Blažek, and Vladimíra Volná. 2016. Transboundary air-pollution transport in the Czech-Polish border region between the cities of Ostrava and Katowice. Central European Journal of Public Health 24(Suppl):S45–S50. https://doi.org/10.21101/cejph.a4532.

    Article  Google Scholar 

  • Crescenzi, Riccardo, and Mara Giua. 2016. The EU cohesion policy in context: does a bottom-up approach work in all regions? Environment and Planning A 48(11):2340–2357. https://doi.org/10.1177/0308518x16658291.

    Article  Google Scholar 

  • Czarnitzki, Dirk, Cindy Lopes Bento, and Thorsten Doherr. 2011. Counterfactual impact evaluation of cohesion policy: work package 2: examples from support to innovation and research. Final report, ZEW Gutachten/Forschungsberichte, KatholiekeUniversiteit, Leuven. https://www.econstor.eu/bitstream/10419/110541/1/823268233.pdf. Accessed 4 Jan 2019.

    Google Scholar 

  • Dvouletý, Ondřej, and Ivana Blažková. 2017. Are publicly supported companies in the Czech food and drink industry performing better? Initial findings from the microdata. Paper presented at the Innovation management, entrepreneurship and sustainability (IMES 2017), Prague.

    Google Scholar 

  • Dvouletý, Ondřej, Jan Čadil, and Karel Mirošník. 2019. Do firms supported by credit guarantee schemes report better financial results 2 years after the end of intervention? The B.E. Journal of Economic Analysis & Policy 19(1):1–20. https://doi.org/10.1515/bejeap-2018-0057.

    Article  Google Scholar 

  • Dąbrowski, Marcin. 2008. Structural funds as a driver for institutional change in Poland. Europe-Asia Studies 60(2):227–248. https://doi.org/10.1080/09668130701820101.

    Article  Google Scholar 

  • Dąbrowski, Marcin. 2014. EU cohesion policy, horizontal partnership and the patterns of sub-national governance: insights from central and eastern Europe. European Urban and Regional Studies 21(4):364–383. https://doi.org/10.1177/0969776413481983.

    Article  Google Scholar 

  • EC. 2011. Ex post evaluation of cohesion policy interventions 2000–2006 financed by the Cohesion Fund (including former ISPA): Work Package C—cost benefit analysis of environment projects ed. Brussels: Directorate-General Regional Policy.

    Google Scholar 

  • EC. 2014. Description of the EU Cohesion Policy. http://ec.europa.eu/regional_policy/en/policy/what/investment-policy/. Accessed 27 Jun 2016.

  • EC. 2015. Total allocations of Cohesion Policy 2014–2020* (million €). http://ec.europa.eu/regional_policy/en/funding/available-budget/. Accessed 01 Nov 2015.

  • EEA. 2019. Air pollutant emissions data viewer. https://www.eea.europa.eu/data-and-maps/dashboards/air-pollutant-emissions-data-viewer-1. Accessed 01 Nov 2015.

  • Einiö, Elias. 2014. R&D subsidies and company performance: evidence from geographic variation in government funding based on the ERDF population-density rule. Review of Economics and Statistics 96(4):710–728. https://doi.org/10.1162/REST_a_00410.

    Article  Google Scholar 

  • Ferraro, Paul J. 2009. Counterfactual thinking and impact evaluation in environmental policy. New Directions for Evaluation 2009(122):75–84. https://doi.org/10.1002/ev.297.

    Article  Google Scholar 

  • Ferry, Martin. 2009. Cohesion policy evaluation systems in the Visegrad states: an overview. In Evaluation systems in the Visegrad member states, ed. Stanisław Bienias, Iwona Lewandowska. Warsaw: Ministry of Regional Development.

    Google Scholar 

  • Frondel, Manuel, and Christoph M. Schmidt. 2005. Evaluating environmental programs: the perspective of modern evaluation research. Ecological Economics 55(4):515–526. https://doi.org/10.1016/j.ecolecon.2004.12.013.

    Article  Google Scholar 

  • Garshick, Eric. 2014. Effects of short- and long-term exposures to ambient air pollution on COPD. Eur Respir J 44(3):558–561. https://doi.org/10.1183/09031936.00108814.

    Article  Google Scholar 

  • GEFRA, and IAB. 2010. Ex post evaluation of Cohesion Policy programmes 2000–2006 financed by the European Regional Development Fund work oackage 6c: Enterprise support—an exploratory study using counterfactual methods on available data from Germany. Final Report: Executive Summary, July 2010.

    Google Scholar 

  • Gertler, Paul J., Sebastian Martinez, Patrick Premand, Laura B. Rawlings, and Christel M. J. Vermeersch. 2016. Impact evaluation in practice, 2nd edn., Washington: World Bank.

    Book  Google Scholar 

  • Head, Brian W. 2016. Toward more “evidence-informed” policy making? Public Administration Review 76(3):472–484. https://doi.org/10.1111/puar.12475.

    Article  Google Scholar 

  • Hoerner, Julian, and Paul Stephenson. 2012. Theoretical perspectives on approaches to policy evaluation in the EU: the case of cohesion policy. Public Administration 90(3):699–715. https://doi.org/10.1111/j.1467-9299.2011.02013.x.

    Article  Google Scholar 

  • Højlund, Steven. 2014. Evaluation use in the organizational context—changing focus to improve theory. Evaluation 20(1):26–43. https://doi.org/10.1177/1356389013516053.

    Article  Google Scholar 

  • Holvoet, Nathalie, Dirk Van Esbroeck, Lisbeth Inberg, Lisa Popelier, Bob Peeters, and Ellen Verhofstadt. 2018. To evaluate or not: evaluability study of 40 interventions of Belgian development cooperation. Eval Program Plann 67:189–199. https://doi.org/10.1016/j.evalprogplan.2017.12.005.

    Article  Google Scholar 

  • Jankauskas, Vytautas, and Steffen Eckhard. 2019. International Bureaucracies as Strategic Actors: How the Better Regulation Reform Strengthens the European Commission. Politische Vierteljahresschrift 60(4). https://doi.org/10.1007/s11615-019-00189-3

  • King, Jean A., and Marvin C. Alkin. 2018. The centrality of use: theories of evaluation use and influence and thoughts on the first 50 years of use research. American Journal of Evaluation 40(3):431–458. https://doi.org/10.1177/1098214018796328.

    Article  Google Scholar 

  • Kurtz, C. F., and D. J. Snowden. 2003. The new dynamics of strategy: sense-making in a complex and complicated world. IBM Systems Journal 42(3):462–483. https://doi.org/10.1147/sj.423.0462.

    Article  Google Scholar 

  • Mastenbroek, Ellen, Stijn van Voorst, and Anne Meuwese. 2016. Closing the regulatory cycle? A meta evaluation of ex-post legislative evaluations by the European Commission. Journal of European Public Policy 23(9):1329–1348. https://doi.org/10.1080/13501763.2015.1076874.

    Article  Google Scholar 

  • Medeiros, Eduardo. 2017a. Cross-border cooperation in inner Scandinavia: a territorial impact assessment. Environmental Impact Assessment Review 62:147–157. https://doi.org/10.1016/j.eiar.2016.09.003.

    Article  Google Scholar 

  • Medeiros, Eduardo. 2017b. European Union Cohesion Policy and Spain: a territorial impact assessment. Regional Studies 51(8):1259–1269. https://doi.org/10.1080/00343404.2016.1187719.

    Article  Google Scholar 

  • Meuwese, Anne. 2012. Impact assessment in the European Union: the continuation of politics by other means. In Sustainable development, evaluation and policy-making: theory, practise and quality assurance, ed. A. von Raggamby, F. Rubik, 141–149. Cheltenham, UK: Edward Elgar.

    Google Scholar 

  • Ministry of Environment. 2009. The Operational Programme Environment for the period 2007–2013. Prague: Ministry of Environment.

    Google Scholar 

  • Mouqué, Daniel. 2012. What are counterfactual impact evaluations teaching us about enterprise and innovation support? A series of short papers on regional research and indicators. Brussels: European Commission.

    Google Scholar 

  • Mueller, Christoph E., and Maria Albrecht. 2016. The future of impact evaluation is rigorous and theory-driven. In The future of evaluation: global trends, new challenges, shared perspectives, ed. Reinhard Stockmann, Wolfgang Meyer, 283–293. New York: Palgrave Macmillan.

    Google Scholar 

  • Olejniczak, Karol, Tomasz Kupiec, and Kathryn Newcomer. 2017. Learning from evaluation – the knowledge users’ perspective. Evaluation Theory and Practice 5(2):49–74.

    Google Scholar 

  • Pelucha, Martin, and Viktor Kveton. 2017. Evaluation culture within institutional and methodological context: the case of EU Structural Funds in the Czech Republic. Evaluation Theory and Practic 5(2):1–26.

    Google Scholar 

  • Polverari, Laura. 2016a. 2014–2020 EU cohesion policy: Results orientation through better monitoring. European Structural and Investment Funds Journal 4(1):26–34.

    Google Scholar 

  • Polverari, Laura. 2016b. The new ambitions for 2014–2020 European Structural and Investment Funds evaluation: pouring water in a leaking container? European Structural and Investment Funds Journal 4(2):59–67.

    Google Scholar 

  • Potluka, Oto, and Jan Brůha. 2013. Zkušenosti s kontrafaktuální dopadovou evaluací v České republice. Evaluation Theory and Practice 1(1):53–68.

    Google Scholar 

  • Potluka, Oto, Jan Brůha, and Lenka Slavíková. 2016a. Plnění ambiciózních cílů po kapkách: evaluace dopadů environmentálních investic na kvalitu vod. Evaluační teorie a praxe 4(1):1–18.

    Google Scholar 

  • Potluka, Oto, Jan Brůha, Martin Špaček, and Lucie Vrbová. 2016b. Counterfactual impact evaluation on EU Cohesion Policy interventions in training in companies. Ekonomický časopis 64(6):575–595.

    Google Scholar 

  • Potluka, Oto, Martin Špaček, and Jiří Remr. 2017. NGOs as partners: Obstacles in the EU Cohesion Policy? Ekonomický časopis 65(8):715–736.

    Google Scholar 

  • Quinn, Robert E., and Scott Sonenshein. 2008. Four general strategies for changing human systems. In Handbook of organization development, ed. Thomas G. Cummings, 69–78. Thousand Oaks: SAGE.

    Google Scholar 

  • Radaelli, Claudio M. 2018. Halfway through the better regulation strategy of the Juncker Commission: what eoes the evidence say? JCMS: Journal of Common Market Studies 56:85–95. https://doi.org/10.1111/jcms.12768.

    Article  Google Scholar 

  • Rist, Ray C., and Nicoletta Stame. 2017. From studies to streams: managing evaluative systems. In Comparative policy evaluation, ed. Ray Rist, C. Abingdon. Transaction Publishers: Routledge.

    Google Scholar 

  • Rowe, Andy. 2019. Sustainability-ready evaluation: a call to action. New Directions for Evaluation 2019(162):29–48. https://doi.org/10.1002/ev.20365.

    Article  Google Scholar 

  • Rubin, Donald B. 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology 66(5):688–701.

    Article  Google Scholar 

  • Schoenefeld, Jonas J., and Andrew J. Jordan. 2019. Environmental policy evaluation in the EU: between learning, accountability, and political opportunities? Environmental Politics 28(2):365–384. https://doi.org/10.1080/09644016.2019.1549782.

    Article  Google Scholar 

  • Schoenefeld, Jonas J., Mikael Hildén, and Andrew J. Jordan. 2018. The challenges of monitoring national climate policy: learning lessons from the EU. Climate Policy 18(1):118–128. https://doi.org/10.1080/14693062.2016.1248887.

    Article  Google Scholar 

  • Slavik, Jan, Oto Potluka, and Kristyna Rybova. 2017. Subsidies in waste management: Effective instruments or a cul-de-sac of European structural policies? Waste Manag 65:1–2. https://doi.org/10.1016/j.wasman.2017.05.040.

    Article  Google Scholar 

  • Slavík, Jan, and Kristýna Rybová. 2016. Institutionelle Analyse des Bioabfallmanagements in der Tschechischen Republik. Müll und Abfall 4:168–172.

    Google Scholar 

  • Smismans, Stijn. 2015. Policy evaluation in the EU: the challenges of linking ex ante and ex post appraisal. European Journal of Risk Regulation 6(1):6–26. https://doi.org/10.1017/s1867299x00004244.

    Article  Google Scholar 

  • Strassheim, Holger, and Pekka Kettunen. 2014. When does evidence-based policy turn into policy-based evidence? Configurations, contexts and mechanisms. Evidence & Policy: A Journal of Research, Debate and Practice 10(2):259–277. https://doi.org/10.1332/174426514x13990433991320.

    Article  Google Scholar 

  • UN. 2015. Transforming our world: The 2030 agenda for sustainable development, A/RES/70/1.

    Google Scholar 

  • Velická, Helena, Vladimíra Puklová, Josef Keder, Marek Brabec, Marek Malý, Martin Bobák, Bohumil Kotlík, Vítězslav Jiřík, Vladimír Janout, and Helena Kazmarová. 2015. Asthma exacerbations and symptom variability in children due to short-term ambient air pollution changes in Ostrava, Czech Republic. Central European Journal of Public Health 23(4):292–298.

    Article  Google Scholar 

  • White, Howard. 2009. Theory-based impact evaluation: principles and practice, 3ie Working Paper, International Initiative for Impact Evaluation, June 2009.

    Google Scholar 

  • White, Howard. 2013. An introduction to the use of randomised control trials to evaluate development interventions. Journal of Development Effectiveness 5(1):30–49. https://doi.org/10.1080/19439342.2013.764652.

    Article  Google Scholar 

  • Yorifuji, Takashi, Etsuji Suzuki, and Saori Kashima. 2014. Hourly differences in air pollution and risk of respiratory disease in the elderly: a time-stratified case-crossover study. Environmental Health 13(64):1–11. https://doi.org/10.1186/1476-069X-13-67.

    Article  Google Scholar 

Download references

Acknowledgements

I would like to thank Jonas Schoenefeld and Paul Stephenson for their helpful comments and recommendations on earlier drafts of the manuscript. Moreover, I thank two anonymous reviewers for their valuable comments.

Author information

Authors and Affiliations

  1. Basel, Switzerland

    Oto Potluka

  2. Prague, Czech Republic

    Oto Potluka

Authors
  1. Oto Potluka
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oto Potluka.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Potluka, O. Overcoming Political Resistance to Evaluating the Environmental Impacts of EU Cohesion Policy. Polit Vierteljahresschr 60, 763–784 (2019). https://doi.org/10.1007/s11615-019-00210-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11615-019-00210-9

Keywords

Schlüsswörter

Search

Navigation