1 Introduction

The European Commission (EC)Footnote 1 defines research infrastructures (RIs) as facilities, resources and services that are used by the research communities to conduct research and foster innovation in their fields [1]. They include:

  • Major scientific equipment or sets of instruments;

  • Knowledge-based resources such as collections, archives or scientific data;

  • E-infrastructures, such as data and computing systems and communication networks;

  • Any other infrastructure of a unique nature essential to achieving excellence in research and innovation.

This definition is now internationally accepted (e.g. by the OECD’s GSF,Footnote 2 ESFRIFootnote 3 and the ERIC Council RegulationFootnote 4). Moreover, RIs can be:

  1. 1.

    Single-sited facilities (unified single body of equipment at one single physical location);

  2. 2.

    Distributed facilities (a network of distributed resources: instrumentation, collections, archives, and scientific libraries);

  3. 3.

    Virtual facilities (e.g. ICT based system for scientific research, including high-capacity communication networks, and computing facilities providing services electronically, also known as e-infrastructuresFootnote 5), or

  4. 4.

    Mobile facilities (vehicles designed for scientific research).

2 The RI-PATHS Project

The growing importance placed on the role of RIs, both in Europe and globally, as a basis for conducting research, but also for research-based education, is evident from the increasing public funding allocated to the design, development, operation and upgrading/renewal of RIs. Under the EC’s Horizon 2020 research and technological development framework programme (RTD FP), a key activity line is focused on research infrastructures and aims to “endow Europe with world-class research infrastructures that are accessible to all researchers in Europe and fully exploiting their potential for scientific advancement and innovation. The total budget for this activity in Horizon 2020 is €2.488 million.Footnote 6 In addition, during the period 2014–2020, the European Structural and Investment Funds (ESIF) are investing €6.931 million (out of a total planned expenditure by the European Union (EU) Member States of €10.131 million)Footnote 7 in public investments in favour of research and innovation infrastructures through the European Regional Development Fund (ERDF).

The overall scale of funding is thus significant and concerns investments in both European level RIs (ESFRI projects such as ELI, ESS, etc.) and national and regional scale RIs that serve a broad-based community of researchers and users (including business users, etc.). As the OECD [2] underline, the level and range of types of investments in RIs leads to a diverse group of stakeholders’ (ranging from national and regional public authorities, funders including charitable foundations, RI managers, scientific and industrial users and special interest groups and ultimately civil society) with (differing) interests in the topic and outcome of RI impact assessments.

In this context, the RI-PATHS projectFootnote 8 aims to provide policy makers, funders and RI managers the tools to assess RI impact on the economy and their contribution to resolving societal challenges, etc. The goal is to improve the understanding of the long-term impact pathways [3] of the various types of RIs operating in Europe, and, indeed, internationally. The project is being implemented over a 30-month period beginning in January 2018 and ending in June 2020. The activities include a review of the state of the art, consultation on the needs and practices of the RI community concerning IA, a phase of co-design with RI managers and stakeholders of an IA framework and the piloting of the IA framework by, at least, the four partner RIs (CERN, DESY, ALBA and ELIXIR). The project builds on and has ensured interaction with all relevant past and on-going initiatives including those by the OECD GSF, various ESFRI working groups, the ACCELERATEFootnote 9 project, etc.

Most approaches to date have sought to undertake specific type of impact assessments often at specific points in the lifecycle of an RI, including, e.g., cost-benefit analysis at the pre-investment stage or before an expansion or upgrading, impact on scientific excellence or the attraction of funding (from competitive programmes or industrial partners) during the operational phase, impact on suppliers through procurement, impact on human resources through training, doctoral research, etc. The RI-PATHS team acknowledge that a single ‘one-fits-all-model’ to assessing socio-economic impact is unlikely to meet the needs of the RI stakeholders. Rather the effort is focused on developing an IA framework adopting a modular approach with a generic core model and more detailed sub-models of IA. Moreover, the IA framework is designed to cover the whole life-cycle of RI development, operations and ultimately decommissioning.

The OECD [2] argued that impact assessment should be connected to the strategic objectives and mission of each RI. They proposed six standard objectives that they considered cover all the main dimensions of impact:

  • Be a national or world leading scientific RI and an enabling facility to support science.

  • Be an enabling facility to support innovation.

  • Become integrated in a regional cluster/in regional strategies/Be a hub to facilitate regional collaborations.

  • Promote education outreach and knowledge transfer.

  • Provide scientific support to public policies.

  • Provide high quality scientific data and associated services.

  • Assume social responsibility towards society.

3 A taxonomy for RIs

Taking account of the need to frame impact assessment in the context of the overall mission an RI has set itself (or in many cases has had set for it in the context of national, European or international funding), the RI-PATHS project developed [4] a taxonomy of types of RIs based on their mission and type of research conducted at the RI or research services provided to users (see Table 1).

Table 1 Taxonomy of RIs by mission/type of research conducted
Full size table

The taxonomy was tested during the consultation phase of the project with RI stakeholders and managers. Overall, it was considered to be a useful framework as it covers a wide range of organisations with different objectives (60% of respondents of a survey of RI managers said it fits their case well). At the same, it was recognised that it may be difficult for many RIs to place themselves into just one category. Hence, the typology served as a framework (during the participatory workshops run by the project) to test the hypothesis that there are distinct pathways to impact depending on the level of interaction with users. To this end, a broader distinction can be made between:

  1. (a)

    RIs that offer services and perform own research activities;

  2. (b)

    RIs that are exclusively or primarily user/services oriented.

In line with OECD [5], we define impact as any long-term effect, whether intended, unintended, positive, negative, direct or indirect produced by an intervention. They are the ultimate changes produced in the society by means of a given action or investment decision. Having an overarching list of impact categories can allow a RI to identify where it is not having impacts, as well as where it is having some. This is an important issue for accountability and advocacy, since it allows RIs to investigate more closely why, for instance, it is ‘failing’ to achieve desired results under certain types of impacts.

A distinction can be made between impacts caused by the RI pursuing its core science mission and impacts caused by the RI being a socio-economic actor. In broad terms, these are both socio-economic impacts resulting from the operation of an RI. The former can be e.g. contribution to short-term and/or long-term problem solving, qualification of scientists, impacts on innovation and productivity in the economy, opening up of new perspectives in the policy discourse and outreach and popularisation of knowledge in society. Examples of the latter include employment effects, wages paid and multipliers, qualified procurement effects with impact on innovation, procurement of standardised, off-the-shelf goods and qualification effects for technical staff.

Demonstrating the causal pathway between investment in RIs and the wider socio-economic impacts can be an intricate, complex and labour-intensive task. This suggests the need for a framework that classifies the diverse impact pathways. An IA framework should provide the opportunity to visualise the potential pathways to impacts or the forms of impact that exist. Visualisation can also help understand the system by which impacts are produced and thus improve it.

Accordingly, a core element of the RI-PATHs project has been on co-creating with the RI community a logical model that links RI activities to outputs, outcomes, longer-term results and finally impacts [6]. As described in Fig. 1, the IA framework has been developed in two stages via a series of participatory co-design workshops, followed by a validation workshop and piloting in the four partner RIs before the launch of a final impact assessment framework.

Fig. 1
figure 1

Timeline of the RI-PATHS project

Full size image

4 Discussing Preliminary Results

The findings of the first round of workshops underlined that there are three main lines of inquiry relevant for all types of RIs (even if each of them may be more or less strongly associated with a specific type).

  • Quantifiable impacts that can be captured through quantitative metrics available e.g. in economic analysis: In some areas, impacts can be determined reliably based on economic modelling. Areas to be thus covered include the transmission of socio-economic effects through employment, (qualified) procurement, some dimensions of training as well as RIs ‘gravitational effect’ on visitors. This approach takes and justifies assumptions but does not further reflect on specific pathways in the process. It has the clear advantage to produce aggregate figures in monetary terms and easy-to-process results for the political discourse.

  • Non-quantifiable impacts the assessment of which can, however, be supported by quantitative means (e.g. social network analysis, dedicated surveys): Some impacts can be characterised through quantities but not suitably translated to monetary terms and aggregated. In these areas actor-based analysis can chart patterns of interaction between RIs and their socioeconomic environment, most prominently their users. It can quantify these interactions and to the extent possible attribute specific effects to them. Thus, it can help establish not only the socio-economic, but also the societal contribution of RIs based on the properties of the network that they have built around them and the known impact pathways that they can be demonstrated to trigger.

  • Complex network effects captured through exploratory approaches which require qualitative approaches such as narrative-based case studies: The analysis of network effects has been highlighted as an area of particular relevance for distributed RIs. It has so far not been intensively explored and will require more work to establish methodological foundations. Ideally, it should be possible to outline a core set of quantitative indicators for this area, e.g. in the area of human capital. In many other areas, this type of analysis will look at complex issues that quantitative approaches fail to satisfactorily capture.

During the first round of RI-PATHS participatory workshops four categories of impacts were used to frame the discussions: (1) Direct economic impact, (2) Human resources impact, (3) Social/societal impact and (4) Policy impact (see Fig. 2). During the workshops, these four broad categories were used to identify twelve stylised impact pathways that help to measure the multi-faceted impact of research infrastructures [7].

Fig. 2
figure 2

Four main types of impact of RIs

Full size image

The goal of the second round of workshops was to engage the RI community in the identification of suitable indicators and metrics that can, in practice, serve to measure the various impacts identified in the first round. As a result of the discussions, the RI-PATHS team decided to re-phrase the initial set of pathways as follows:

Impacts as a result of RIs Enabling Science (i.e. performing their primary mission)

  • Employment & standardised procurement

  • Publication-citation-recognition (incidental spillovers)

  • Technology transfer & licensing (supported spillovers)

  • Learning- and training-through-specialised procurement

  • Generic learning- and training-through-usage

Impacts as a result of RIs Supporting Problem Solution (by providing services to users)

  • Interactive industrial problem solution

  • Interactive public sector problem solution

  • Provision of specifically curated/edited data to industry

  • Provision of specifically curated/edited data to public sector

  • Other interactive societal problem solution

Impacts through shaping future trends in Science and Society

  • Changing fundamentals of research practice

  • Contribution to formal standards in science

  • Creating and shaping scientific networks and communities

  • Creating and shaping networks between science and society

  • Communication, outreach and engagement.

5 Conclusion

The proposed conceptual framework (see Fig. 3) fulfils two central functions:

  • First, it defines impact domains where RI may play a role. While political expectations may shift, findings from the participatory workshops and surveys have confirmed that IA should no longer be solely motivated from a purely compliance-oriented perspective. Increasingly, many RIs are articulating their mission to contribute to broader societal goals.

  • Second, the model helps to define the logic of impact causation, or impact pathways, through which different types of impact materialise. Understanding these pathways can assist decision making by RI managers in terms of the launch of or intensification of specific internal activities or through the collaboration and engagement with external partners.

Fig. 3
figure 3

General conceptual approach of the RI-PATHS Framework

Full size image

Given this overall framework, the second round of workshops were also used to assess and refine a long-list of indicators down to a workable set that can be used to track the various types of impacts and the inter-relations between the impact pathways [8]. In the final step of the project, this approach will be piloted, then refined into an operational guidance that will provide RI managers and stakeholders with an easy-to-use framework that helps them identify the sort of questions an IA exercise should answer and what data needs to be collected.

Moreover, the EC has stated that the RI-PATHS framework will guide the future IA of ESFRI projects and the project team expect that the guidance (to be licenced under Creative Commons) will be further developed by practitioners in the coming years.