Keywords

Introduction

In order to analyse the added value promoted by the EU urban initiatives, we must define their essential characteristics, both from an analytical and an operational perspective, which will allow us to measure them and conduct comparative analyses. Our analytical proposal is defined in Chapter 1. In this chapter, the research strategy proposed is presented. How have EU urban initiatives been previously analysed? What research methods have been used?; what issues do these strategies allow for analysing?; can these strategies provide comparative policy evidence about the expected added value of the urban dimension of the EU cohesion policy on urban policies design and implementation?

In the first section of this chapter, we briefly review the main methodological perspectives used by previous research to analyse the nature—and changes—of EU urban initiatives and propose an alternative approach as regards these questions: the comparative urban policy portfolio analysis (CUPPA). This approach will then be applied to provide evidence on some central aspects of the urban dimension of EU cohesion policy and its added value for urban policies in chapters included in the first part of this book. A brief introduction to this objective is done in the second section of this chapter.

Analysing EU Integrated Urban Initiatives: Main Methodological Perspectives

From a methodological perspective, research on the nature of integrated urban development programmes and initiatives promoted by the European Union can be grouped into three broad approaches also used commonly to study other public policies: the normative approach; the analysis of programmes; and the study of policy domains (Howlett et al., 2006). These are complementary perspectives that analyse aspects related to policy goals and implementation preferences, the two main dimensions of public policies. They, therefore, help highlight relevant aspects of the policy framework established by a policy, as well as the specific projects developed within that framework.

The first and perhaps the most common approach is to analyse the normative or programmatic production of urban development initiatives promoted by the EU. This has allowed researchers to define the basic foundations of such initiatives, such as identifying the URBAN Acquis on which integrated urban development strategy in the EU has been based, as well as their evolution from integrated urban regeneration to the idea of sustainable urban development (Cotella, 2019; Dood, 2011; European Commission, 2014, 2015; Scheurer & Haase, 2018; Medeiros, 2019). This approach has shown significant changes between programming periods of the EU’s cohesion policy and differences between countries (or regions). However, it does not show the variety found in the application of integrated strategy proposed by the EU in different territorial contexts, particularly at the scale of the territorial target to which they are applied (neighbourhoods, cities, urban functional areas, …). Furthermore, these studies are usually based on an ‘open’ or ‘conventional approach’ to documentation analysis or at least do not report the application of a systematic and reproducible methodology for the development of comparative analysis, as the ‘direct approach’ in content analysis tries to ensure (Hsieh & Shanon, 2005).

The programme perspective entails an extensive analysis of projects implemented within the framework of a programme or public policy. Examples of this approach are the ex-post assessments of urban initiatives carried out by the European Commission (European Commission, 2002, 2010, 2016) or the information provided by the STRAT-Board elaborated by the Joint Research Centre for urban initiatives included in ESIF during the 2014–2020 period, or analyses specific countries or regions; for instance, Medeiros (2020) or Partidário and Nunes (2004) in Portugal and De Gregorio (2017) in Spain. Typically, this perspective provides information on the content of projects through planned or implemented budgetary spending on the ‘official’ objectives, policy areas or priorities established in their respective regulatory frameworks and calls for proposals. This information facilitates the analysis of project goals and, through aggregation, of programme goals, although it can present some difficulties for their comparative analysis. Firstly, the volume of financial resources required to implement actions within different objectives varies without necessarily taking account of their importance in the strategy pursued by the projects. One example would be the classic distinction between actions involving large investments linked to infrastructures or interventions in the physical space, as opposed to those aimed at providing services to specific groups (Sharp, 1990). This may affect both the analysis of projects within the framework of a single programme and the comparative study of several programmes. Secondly, there have been changes in the designation of the different areas of action (objectives, thematic priorities, …) between programmes or programming periods. Therefore, even if there is a similarity in their contents (pursued goals or the strategies proposed to achieve them), they cannot be compared based on their official definitions. Finally, this perspective does not usually provide information on the instruments or ‘policy tools’ used by projects, or at least with sufficient detail to enable their comparative analysis. Therefore, we might obtain details about their intended aims, but somewhat less on how they intend to achieve them. In other words, just one of the two primary dimensions of public policy (goals, but not policy instruments).

The policy domain perspective, in contrast, focuses on conducting a detailed study of projects to know which actors they involve, their activities and their relationships of interaction and influence. In this approach, the main objective is to analyse governance processes based on the structure of their policy networks and the roles played by different actors according to the public policy and policy issue under analysis. These analyses could provide detailed information about decisional and implementation processes similar to the classical ‘community power studies’ (Fedeli & Doria, 2006). However, it is a very costly approach to implement, which might explain why this is the least widespread perspective, and why its most common application is the development of case studies or the comparative study of some cases (Navarro & Rodriguez-Garcia, 2015; Rodriguez-Garcia & Navarro, 2016).

The above three perspectives or methods have been providing very valuable—and complementary—information to understand certain essential aspects of the policy framework surrounding the urban development initiatives promoted by the EU, as well as some features of the design and implementation of their projects at a local level. However, as we have seen, they present certain limitations for developing comparative analyses (see Table 2.1). One possible alternative, not without limitations either, is the strategy proposed here: the CUPPA or comparative urban policy portfolio analysis (Navarro & Rodriguez-Garcia, 2020). This strategy involves ‘reconstructing’ the nature of urban policies based on two premises. First, the analysis focuses on the concrete projects developed and the specific actions they contain. The unit of observation is not the policies or programmes, but the projects through which they are actually carried out at the local level, as well as the policy measures they include (their minimum unit of planning and implementation), enabling public policies to be analysed ‘from the bottom up’ according to what is actually planned and implemented in each case. Second, analysis is based on analytical categories theoretically grounded in urban policy literature. These categories, as analytical units, are therefore independent of possible changes in the normative or programmatic documentation of the policies or programmes analysed (from the definition of their objectives or their preferences for implementation), thus allowing for comparative analysis between projects, programmes, and policies across different contexts and periods.

Table 2.1 Analysing the design and implementation of sustainable and integrated urban initiatives promoted by the EU: main methodological perspectives
Full size table

Thus, CUPPA proposes a theoretically bottom-up approach to analysing urban policies. It allows for the comparative study of urban initiatives promoted by the EU at different levels taking into account the nested nature of these initiatives within the framework of the EU’s cohesion policy (from policy measures to local projects, national or regional programmes, and the urban initiatives of the EU cohesion policy). Hence, the actual policy frame is analysed, not just the normative or programmatic proposals. This method also allows for conducting cross-sectional and cross-time analyses at different policy levels (policy measures, projects, programmes, regions, state members, and programming periods).

From an operational point of view, CUPPA involves the content analysis of the design documents (goals and policy actions planned) or evaluation documents (policy actions implemented) of all projects developed within the framework of the policies analysed. To this end, the direct content analysis approach is applied using a coding template to analyse policy actions and a template to analyse aspects related to the project as a whole. This methodological approach is common to public policy evaluability assessment and urban policies (Hsieh & Shanon, 2005; Lyles & Stevens, 2014; Trevisan, 2007). The same coding template is applied to each local project (and their policy actions) with categories defined from theoretical perspectives referring to different aspects of the study of urban policies. The analysis provides a hierarchical database from the specific policy actions to local projects, and then, to the programmes and policies. At each policy level, basic aspects of their substantive and procedural dimensions are analysed (e.g. objectives, policy tools, actors involved, coordination mechanisms, evaluation mechanisms, etc.). These aspects could be studied either in terms of policy design, analysing the design documentation, or in terms of policy implementation, analysing the evaluation document that gives an account of the actions implemented and their achievements.

The application of the CUPPA approach depends on the existence of a design document and an evaluation document for each project providing relevant information on central aspects of any public policy, in general, and urban intervention projects in particular; namely: a diagnosis detailing challenges and opportunities; the desired future situation in terms of strategic goals and objectives; the policy actions and policy instruments that will make it possible to achieve these objectives; the processes that will ensure coordination and collaboration between the different actors involved in the projects; as well as a monitoring and evaluation system to supervise the implementation process and show evidence of the achievements and impacts made (Guyadeen & Seasons, 2018; Kaiser et al., 1995; Lyles & Stevens, 2014; Oliveira & Pinho, 2010).

The urban initiatives promoted by the EU are usually explained in a document establishing the vision and strategic actions to be developed in a territory. These documents typically include a diagnosis describing challenges existing in the targeted territory, pursued objectives, policy actions to be implemented, instruments chosen for their management, the promotion of processes of governance and participation, as well as their evaluation systems. In some cases, evaluative reports are made, and these documents include some information about what has been done and their degree of achievement according to established objectives. These sources of information offer us the possibility of studying projects by applying the CUPPA approach.

In the case of the URBAN and URBANA initiatives in Spain, information on the projects (design and evaluation documents) has been provided to us by the Directorate General of Community Funds within the Ministry of Finance. It has not been possible to study the design documents of the URBAN projects developed between 1994 and 1997 and two URBANA projects, since the documentation for these was not retained. It is, therefore, possible to analyse 64 projects at this level. However, it has been possible to analyse all the policy actions implemented in all projects based on their evaluation reports: a set of 611 policy actions and around 2000 specific policy measures for a total of 82 projects. This supposes a detailed dataset to analyse the design and implementation of local projects from a comparative perspective: between projects (within programmes or policies), between programmes (or policies), and different time periods (for instance, Cohesion policy programming periods).Footnote 1

The Nature of EU Urban Initiatives and Their ‘Added Value I’: Quality of Design, Policy Agenda Content, Integrated Strategy, and Policy Theories

Based on the information gathered, the CUPPA methodology has been applied to study different aspects of the urban development initiatives promoted by the European Union in Spain between 1993 and 2014, comparing those implemented under the URBAN I and URBAN II Initiative (1993–2006) with those implemented under the Spanish URBAN Initiative (2007–2014). The aim is to develop three objectives.

The first objective is to provide descriptive policy evidence on four essential aspects of these initiatives. Firstly, we studied the quality of project design from the perspective of policy evaluability. Have the basic dimensions of the projects been adequately defined? Would they be evaluable? Is the evaluation planned? Secondly, we examined the policy agenda (or content of projects) according to the relative importance of different areas or sectors of public policy as challenges, objectives, and actions of the projects. Unlike other studies, we do not consider the presence—or absence—of different policy sectors or their weight in the project budget, but their importance in the strategic vision of each project. Thirdly, we investigate the application of the integrated strategy in the projects, both in terms of their content and with respect to the actors involved, showing it is a different aspect to the diversity or juxtaposition of different objectives, actors, or processes within the same project. Lastly, we analyse the intervention strategy of the projects, understood as policy mixes, through the causal processes and mechanisms that underlie the implemented actions.

The second objective, which focuses on methodological issues, is to provide strategies for the comparative analysis of EU urban initiatives based on the CUPPA approach. For this reason, some sections of the chapters detail how the aspects analysed have been conceptualised and measured to perform comparative analyses. The intention is to show how this approach can be used to analyse integrated and sustainable urban development initiatives promoted by the EU from a multi-scale and comparative perspective that is not afforded by a programme-focused perspective, case studies, or normative analysis. The CUPPA perspective considers the nested and multi-level nature of urban initiatives within the framework of EU cohesion policy, offering the possibility of comparative studies from a multi-scale and longitudinal perspective; and, therefore, the development of the next objective.

The third objective—evaluative in nature—is to provide policy evidence about the added value of these urban policy initiatives through the comparative analysis of the URBAN and URBANA programmes. On the one hand, added value will be understood as a learning process in urban policy design: has the quality of project design improved? and has the integrated strategy been incorporated into projects? On the other hand, added value will be understood as the nature of the policy frame being promoted: what changes are taking place in its substantive dimension?, do they incorporate the idea of sustainable urban development?, are there differences in the problems they seek to solve, their objectives and the way they seek to achieve them?, do these differences indicate changes in the direction of the urban dimension of the EU cohesion policy?

Simple analysis techniques are used to carry out the comparative analysis between the projects included in the URBAN and the URBANA initiatives. In some cases, the ‘effect size’ is also provided. This shows the standardised differences between compared groups, URBAN and URBANA projects here, regardless of the measurement scale used, as well as its confidence interval (Fritz et al., 2012; Lakens, 2013). Specifically, we will use the g indicator proposed by Hedges since, unlike Cohen’s classic d (Cohen, 1988), it considers the difference in the number of cases in the groups being compared (Hedges, 1981; Hedges & Olkin, 1985). According to the general rule established by Cohen (1988), values less than 0.2 would show no effect (or differences); values between 0.2 and 0.4 means a small effect; a medium effect for values between 0.5 and 0.7; and large effects for values greater than 0.8. In addition, to facilitate their practical interpretation, effect size values can be transformed into the U3 indicator (Cohen, 1988), which shows the percentage of projects in a group with values above the average of the group with which it is compared, here URBANA versus URBAN. Thus, the existence of a small effect size value would mean that between 58 and 68% of the URBANA Initiative projects have higher values than URBAN projects (g between 0.2 and 0.49), a medium effect would indicate that this percentage is between 69 and 78 (g between 0.50 and 0.79), and a large effect would be anything above 80% (g = 0.8). This way, the practical significance of the difference between the programmes can be seen more clearly.

Overall, the first part of this text aims to provide evidence on some essential aspects or dimensions of the model of integrated and sustainable urban development that the EU's cohesion policy has promoted since the 1990s in Spain. What has this entailed? How did the policy frame for integrated urban development programmes in Spain involved between 1994 and 2017? Has there been the expected improvement or added value? The evidence will allow us to answer these questions. The following chapters also aim to offer analytical and empirical tools for the comparative analysis of integrated urban development projects, programmes, and policies promoted by the EU, that could be applied to other cases (countries, moments in time,…). The main research questions in the following chapters, the analytical unit—or the specific topic analysed—and the observational unit for each case are summarised in Table 2.2.

Table 2.2 Analysing the integral strategy and added value I (better policies): research question, analytical and observational units
Full size table