1 Introduction

The measurement of the impact of research on society is an extremely relevant matter (Bornmann, 2013). When research is conducted with public funding, the measurement of these impacts is closely related to the need to demonstrate its value to funding entities, the opportunity to leverage funding for future research, and the ability to identify more efficient ways to generate greater impact (Penfield et al., 2014; Walsh et al., 2018). However, in the context of university-industry R&D collaborations (UICs), measuring impact is a complex task due to the heterogeneity of the institutions involved, the diversity of objectives and expected benefits, and different perspectives of each stakeholder since high-value impact for one group may not be the same for another (Fini et al., 2018).

Although a considerable body of scientific literature has addressed the socio-economic impacts of UICs in the past decade (Lima et al., 2021), comprehensive assessments of the broader impacts of collaborative research, known as ‘societal’ impacts, have remained limited (Bornmann, 2013; Galan-Muros & Davey, 2019; Siemieniako et al., 2021; Skute et al., 2019; Tijssen, 2012). These societal impacts are characterized by their macro-level nature, namely encompassing social aspects, as indicated by Siemieniako et al. (2021). The complexity of these impacts is exacerbated by a diffuse boundary that makes it challenging to clearly identify their relation to quality of life, health, or the environment, resulting in ambiguity when determining whether an impact is social, economic, or of another sort (Bornmann, 2013).

The challenge of conceptualizing the impacts of research is not new. In 2011, for example, the Health Economics Research Group organized an international workshop to gather academic and professional views on new pathways for assessing the social impact of research. Most participants agreed on the difficulty of finding a clear concept of social impact that could facilitate such evaluation (Donovan, 2011), an issue that continues to be mentioned in current studies on the impact of research in organizational contexts (Siemieniako et al., 2021).

In addition to the conceptual challenges associated with analyzing impacts, Galan-Muros and Davey (2019) characterized the field of UICs as fragmented, due to the limited linkages between its thematic domains. However, they have endeavored to integrate its elements into a conceptual framework, where UIC impacts are regarded as a central element within the UIC. In the same way, Skute et al. (2019) conducted a bibliometric study to map the research field of UIC and acknowledge the importance of analyzing the economic and social impacts generated by these collaborations at regional and national levels.

In the general context of UICs, the impact is defined as the outcome indirectly experienced by individuals, institutions, and society (Galan-Muros & Davey, 2019), or as the direct or indirect result that influences stakeholders, including society (Albats et al., 2018). In this study, impact is defined as a positive or negative change originated in the UIC context that can directly or indirectly affect individuals, organizations, communities and society in general (Siemieniako et al., 2021). The impacts caused by UICs can produce economic, environmental, health, cultural, political effects at the macro level and on the quality of life, stemming from the creation of new or improved products and services based on scientific knowledge (Fini et al., 2018).

UICs require support mechanisms, i.e. management (Fan et al., 2019), political, structural, operational, and strategic mechanisms to ensure that research is relevant to society (Galán-Muros et al., 2017), and is capable of creating monetary and non-monetary impacts that converge towards the boundaries collectively (Audretsch et al., 2019).

In fact, there are several key factors that may constrain or drive the impacts of UICs. The absence of shared objectives among universities, science, and businesses is a significant limiting factor (Issabekov et al., 2022), demanding sustainable strategies to maintain common interests over time (de Freitas et al., 2014). Factors such as company size, sector, commitment to digitization (Marra et al., 2022) and level of trust emerge as crucial drivers of innovation and future collaborative projects (Vega-González et al., 2012). Information asymmetry within the UIC is pointed out as a critical factor hindering the commercialization of university patents (Xiaojuan & Hongda, 2021).

The impacts of UICs are also influenced by the absorptive capacity of companies: companies with high absorptive capacity have a unique competitive advantage, adapting to changes in the environment and fostering innovation (Tian et al., 2021). In fact, when funding collaborative projects, governments tend to favor companies with high absorptive capacity, underscoring the relevance of this factor for the success of UICs (Cui et al., 2022). The synchrony between regional innovation and economic development fosters the correlation between basic research and market demand, leading to higher UIC impact (Cui & Li, 2022). Finally, institutional factors and structural conditions, such as economic cycles, impact UIC scientific production. For example, in a crisis, there may be an interest in signaling potential scientific areas that promote UICs and co-publication production (Azagra-Caro et al., 2018).

The present study identifies a gap in understanding how UICs impact society (Di Maria et al., 2019; Jones & Corral de Zubielqui, 2017; Nugent et al., 2022). To address this gap, a systematic literature review was conducted, by thematically analysing 92 studies published between 2000 and 2022. The current paper seeks chiefly to identify and categorize the types of impacts of UICs from the perspectives of universities, industry, and society. Additionally, it examines the challenges of measuring these impacts and identifies the strategies employed to overcome such challenges.

The literature review helped us identify a set of 25 impacts of UICs, which are subsequently classified into six categories. Some of the challenges in measuring these impacts are related to their intangible or transient nature, that is, their ability to appear, disappear, or transform from positive to negative across the collaborative lifecycle (Perkmann et al., 2011), as well as the complexity of dealing with the various causes that can explain their origin (Fini et al., 2018). Such intrinsic characteristics of impact make them hard to measure, the perspectives of the agents involved in the collaboration thus being a crucial element in the measurement process (Penfield et al., 2014). Finally, some strategies are presented to overcome the challenges of measuring impacts of UICs. Many of these strategies are utilized in empirical studies, while others are theoretical guidelines that can be implemented in future studies.

The main contribution of this article is to consolidate insights from the past two decades regarding the impacts of UICs, subsequently presenting key elements of the process in a single framework that can serve as a basis for the development of future impact measurement methodologies. Additionally, it provides thoughts on the need to advance in measuring more comprehensive impacts, considering not only the academic or industrial community but also other social groups that may be affected by the collaboration. It also encourages a deeper analysis of key factors that may restrict the realization of impacts.

The rest of this paper is structured as follows. Next, the context of UICs and their impact on society is presented. Then the research methodology employed is detailed. Subsequently, it delves into the findings, an unfolds in a comprehensive discussion, concluding with insights into potential future research directions.

2 Background

2.1 UIC

The literature defines UIC in general terms as a type of alliance that benefits innovation performance significantly (Wirsich et al., 2016), resulting in a positive impact on R&D participation and learning opportunities for the company involved (Scandura, 2016). UIC is also described as an interactive relationship that aims to enhance competitive advantages through trust, commitment, and access to each partner's resources, aimed at producing a social impact (Galan-Muros & Davey, 2019), which can be formal in nature when delivered by explicit contracts, or informal when focused on personal interactions using trust as a prerequisite for collaboration (Apa et al., 2021).

More specifically, UICs are defined as agreements between the university and the industry with the purpose of conducting joint research. Some of the R&D activities included in this type of collaboration are contract research projects, joint publications by industry and university or R&D consulting (Pinto & Fernandes, 2021). A good management system with appropriate mechanisms is thus a critical tool to influence the expected impacts and control the uncertainty underpinning this type of collaboration (Morandi, 2013).

The aforementioned definitions imply that UICs are aligned with the concept of 'Mode 2 of production.' This approach represents a different and interdisciplinary way of generating knowledge between the scientific community and other stakeholders, with the aim of impacting industry, government and society. In this context, knowledge production takes place through a continuous negotiation of interests among the various actors involved (Gibbons et al., 1994). In contrast to 'Mode 1' of production, which focuses on the interests of the academic community and aims to generate high-impact research, 'Mode 2' is considered more suitable for generating socially useful research, albeit with a lower impact factor (Nightingale & Scott, 2007).

Similarly, the literature related to UICs has long considered that the lack of complementarity between industry and academic activities undermines scientific production (Perkmann & Walsh, 2009) or produced low impact factor publications (Abramo et al., 2009). However, recent studies have shown that UICs built upon expected complementarities, such as resources (Zhang et al., 2022), skills, availability of equipment, and task distribution among academic and industry scientists, can increase scientific production and enhance the business activity in the industry (Bikard et al., 2019). Nevertheless, there is a need to understand the impacts of UICs on society (Di Maria et al., 2019; Jones & Corral de Zubielqui, 2017; Nugent et al., 2022).

On the one hand, academics' interest in translating the results of their research into broader benefits for society (Nugent et al., 2022) is justified by funding institutions’ focus on the real contribution of their investments (Penfield et al., 2014). On the other hand, the strong pressures experienced by companies and universities due to the speed of technological change, the quest for more advanced knowledge, the growing cost of research, and the need to address social and economic problems stimulate the creation of UICs worldwide and demand proof of their impact capacity (Ankrah & AL-Tabbaa, 2015).

2.2 UIC impacts

Literature published in the last two decades has chiefly focused its analysis on the industry perspective and generally agrees that business innovation is an important positive impact of UICs (e.g., Apa et al., 2021; Eom & Lee, 2010; Giannopoulou et al., 2019). Political agendas have evolved with the inclusion of science-based technological innovation. However, the absence of reliable quality indicators at the business and technological levels hinders effective guidance for policymakers, limiting broader impacts on society (Tijssen, 2012).

From the perspective of universities, there is a consensus regarding UICs affecting academic productivity, but there are different points of view as regards this effect and its positive or negative nature (Banal-Estañol et al., 2015; Bikard et al., 2019; Perkmann & Walsh, 2009). This lack of consensus has encouraged Garcia et al. (2020) to analyze the impact of UICs on the productivity of academic research in the long-term, due to the ease of managing contract rules between universities, companies and funding agencies. The results confirm that the long-term impact is positive; however, this occurs at decreasing rates, suggesting that the positive effects of UIC on scientific productivity may be constrained over time.

Faced with the complex task of identifying and developing a classification for the different types of impacts, the theoretical field of interorganizational relations offers interesting prospects for the analysis of the impact of research, categorizing it into three levels: micro, mezzo and macro (Siemieniako et al., 2021). The micro level is related to individual aspects in the organization, the mezzo level pertains to aspects that affect specific groups acting within the organization, and the macro level encompasses groups or communities outside of the organization, transcending interorganizational relationships (Siemieniako et al., 2021).

We believe that the interorganizational approach can be applied to the field of UICs, allowing for the evaluation of impact from both an internal and external perspective. At the micro level, it would be possible to consider the impacts experienced directly or indirectly by academics, researchers, students, entrepreneurs, or any other individual involved in the collaborative environment. At the mezzo level, impacts are experienced by research teams, industrial associations, and communities within the collaborative context. Finally, at the macro level, impacts would extend to external communities that are directly or indirectly affected by the collaboration. These communities can encompass various domains such as industry, academia, region or any other group in society (Galan-Muros & Davey, 2019).

Another relevant discussion in the literature addresses the pathways for UICs to generate greater impact on society. One perspective, as proposed by Bornmann (2013), argues that knowledge commercialization is a way to create broader impacts. In other words, when research outcomes are transformed into marketable products, such as consumer goods, medicines, devices, or services, broader impacts are achieved. For example, the Argus II device, an artificial retina resulting from a collaboration between academia, industry, and the government, materialized in socially important innovations (Walsh et al., 2018). However, it is important to mention that UICs generally pursue research objectives through joint R&D activities, and their results usually become intellectual property assets, such as patents, licenses, and sales, which are subsequently traded (Pinto & Fernandes, 2021).

Fini et al. (2018) argue that there is a lack of understanding of how research can impact through commercialization. In this regard, the authors suggest moving away from the emphasis on direct outcomes of commercialization (such as patents and licenses) and understanding commercialization as the process of turning knowledge into useful products or services available on the market. This new approach involves creating direct links between users and performers of R&D activities, which would give rise to collaborative projects targeting user needs and generating higher societal impact (Fini et al., 2018).

2.3 Previous reviews and research gap

Extant literature reviews have widely emphasized the analysis of crucial factors for technology transfer (Da Silva Florencio & De Oliveira, 2022) and collaborative innovation (Sjoo & Hellstrom, 2019). Reviews grounded in case studies have also explored aspects often overlooked in such relationships, such as the choice of partners and the management of stakeholder interactions (Marinho et al., 2020). These review studies frequently consider the analysis of each of these factors at different levels, spanning from the individual to the institutional and academic level (Puerta Sierra et al., 2017).

Other reviews highlight the challenges and motivations faced by universities at both individual and institutional levels, as outlined by Harryson et al. (2007) and Nsanzumuhire and Groot (2020). These aspects assume particular relevance as the academic community interested in collaborating with industry grapples with the challenge of legitimizing their activities within the academic sphere while balancing their responsibilities of teaching, research, and participation in industrial initiatives (Miller et al., 2018).

The relationship between academic engagement and commercialization has also undergone thorough analysis. As discussed by Perkmann et al. (2013) academic engagement is interpreted as a multi-level phenomenon influenced by both individual characteristics and the organizational and institutional context. It serves as a mechanism for resource acquisition by high-performing academics in institutions with limited resources (Perkmann et al., 2013).

Although the aforementioned studies do not specifically focus on analyzing UIC impacts, they recognize the need to address this theme in future research. In contrast, some authors have explored less extensively the impacts arising from knowledge-sharing collaboration (Mascarenhas et al., 2018), as well as the effects of the trilateral relationship between university, industry, and government in regional innovation systems (Lew & Park, 2021).

The systematic review conducted by Lima et al. (2021) leads to a conceptual model classifying the UIC impacts into three categories: economic, social, and financial. This study underscores the social impact of UICs as an emerging field characterized by predominantly exploratory and qualitative research, encompassing various theoretical approaches, albeit still lacking a more robust foundation (Lima et al., 2021). Similarly, qualitative analysis techniques have been employed in literature reviews to identify the economic, institutional, and social benefits of UICs (Ankrah & AL-Tabbaa, 2015), as well as the proposition of impact assessments of public–private partnerships, notably in the biomedical and pharmaceutical sectors (de Vrueh & Crommelin, 2017).

It is worth mentioning that a considerable number of studies have employed quantitative methods with a particular emphasis on the use of econometric models (Apa et al., 2021; Di Maria et al., 2019; Vega-Jurado et al., 2020), structural equations combined with narratives (De Silva et al., 2021) and case studies (Azagra-Caro et al., 2017) to measure the impact of UICs. This trend in the combined use of methods confirms that the type of methodological approach employed in measuring the impacts of research (Bornmann, 2013) remains relevant in the field of UIC. However, the current paper does not intend to delve into the specific methods used to measure the impacts of UICs, which would otherwise extend the length of the paper significantly.

Given that the literature review revealed a gap in measuring the impacts of UICs (Galan-Muros & Davey, 2019), and motivated by studies emphasizing the importance of broadening the scope of UIC impact analysis (Mascarenhas et al., 2018; Miller et al., 2018; Skute et al., 2019) and influenced by the remarkable proliferation of literature related to the emerging impacts of the UIC in recent years, we conduct a systematic literature review that identifies and categorizes the types of UIC impacts. Furthermore, we identify the challenges in measuring these impacts and the strategies that have been employed to address such challenges. We observe that these aspects have not been addressed simultaneously and comprehensively in previous reviews. Therefore, bringing this information into a single framework will serve as a foundation for future empirical research aimed at developing systematic methodologies for measuring the impact of UICs.

The UIC measurement impact framework here proposed aims to provide a structured approach to better understanding the categories, challenges, and strategies related to the measurement of impact across the UIC lifecycle. With the help of this framework, researchers will be able to be more rigorous, careful, and strategic in analyzing the impact in real contexts. Ultimately, our goal is to advance the knowledge and understanding of the impact of UICs on society at large.

3 Research methodology

To understand the process of measuring the impacts of UICs, we have identified (a) types of UIC impacts, (b) categorization of impacts, (c) challenges of measuring UIC impacts, and (d) strategies to overcome these challenges. Following the systematic review process presented by Tranfield et al. (2003), we have divided this process into three phases: phase I, review planning; phase II, identification, and selection of studies; phase III, evaluation of study quality, data extraction (thematic analysis) and presentation of results (Fig. 1).

Fig. 1
figure 1

Systematic literature review phases

Full size image

Each phase was preceded by periodic meetings held by three authors to discuss issues related to the application and follow-up of the protocol. After a cycle of preliminary readings, the keywords selected for the search in Scopus and Web of Science were as follows: ("UIC*" OR "university-industry" OR "industry-university" OR "UBC*" OR "university-business cooperation" OR “public–private” OR “private–public”); ("university*" OR "academic*" OR "higher education"); (“ industry *” OR “enterprise *” OR “company*” OR “firm*”); ("impact*" OR "benefit*"); (“R&D” OR “Innovation”). The expressions were searched in the title, abstract, and keywords of the articles.

The initial process resulted in 1.593 documents which four filters were applied to (type of document, language, year of publication, and fully completed articles). That is, only articles in English published between 2000 and 2022 were searched, considering that from 2000 onwards there were more studies focusing on measuring the impacts of UICs. However, interest in this thematic area began to grow in the last five years, as also evidenced by Lima et al. (2021). Therefore, the danger of omitting relevant studies can be minimized by analyzing recent articles that use previous studies as a basis (Ankrah & AL-Tabbaa, 2015). This filtering procedure eliminated 620 documents, leaving a total of 973 articles. After eliminating duplicates, 665 articles passed the selection and evaluation phase.

Later in the process, two inclusion criteria were applied to the aforementioned 665 articles, incorporating in the analysis those articles that, based on their abstract, provided a positive response to at least one of the following questions: 1) Does the study address the impacts of UIC? 2) Does the study address useful mechanisms or criteria for measuring the impacts of UIC? In some cases, it was necessary to go beyond the abstract to answer these questions. This process resulted in 172 articles being selected for further analysis.

Finally, as our interest was centered on studies that addressed the types of impacts, concepts, ways of measuring them, or key theoretical elements to consider in a measurement process, a detailed reading of each article allowed us to eliminate those whose contribution did not offer the degree of depth necessary for this research, resulting in a final set of 77 articles for the data extraction process. The snowballing strategy was employed in the literature review. The examination of the 77 key articles allowed for the identification of significant theoretical contributions, which guided the review of the relevant references. This process facilitated the inclusion of 15 additional articles, resulting in a more comprehensive understanding of the topic across a total of 92 analyzed articles.

The data analysis process for identifying and categorizing UIC impacts was carried out following two main steps. Firstly, the selected articles were carefully scanned to extract key data, which was organized in a matrix. This information encompassed the author and publication date of the article, the identified impacts, the impacted 'agent' (whether it was the university, society, or industry), and the impacted area (social, economic, technological, environmental, intellectual, or strategic). It is worth noting that most of the literature indicated, either explicitly or implicitly, whether an impact fell into the economic, environmental, technological, social, or intellectual categories. However, for impacts such as reputation, competitiveness, and new collaborations, it was less evident to classify them into a specific category. Nevertheless, given their direct association with an organization's image, they were found 'strategic.' Throughout this data analysis process, in-depth and comprehensive readings of the selected texts were conducted. This immersion in literature was crucial for extracting key data and thoroughly understanding the various impacts identified in the studies selected.

Secondly, we merged some impacts, due to terminological variations when referring to similar impacts. Therefore, a regrouping process was undertaken, resulting in a total of 25 identified UIC impacts. Categorization was carried out by linking the impact's meaning with the corresponding, most affected area, according to the authors' views. To ensure accuracy and consistency in the impact categorization process, researchers met several times to engage in meaningful discussions and achieve a consensus in the categorization of each impact.

The challenges of measuring the impact of UICs and the strategies to overcome such challenges were not explicit in most of the selected articles. We strategically applied a methodological approach of reflexive thematic analysis, which allows for the use of the researcher’s subjectivity in the data analysis process, and for being flexible and recursive, without having to follow a linear process (Braun et al., 2019). Our thematic analysis started with “focused familiarization”, i.e. the documents were analyzed by focusing on two central ideas, "impact measurement challenges" and "strategies to overcome impact measurement challenges". The first phase identified a set of topics connected to each predefined core idea, leading to a second phase that consisted of analyzing and discussing the relationships and interpreting the coherence of each topic. Finally, a third phase fostered a discussion tying up all topics in a comprehensive framework.

4 Results

4.1 Identification and categorization of UIC impacts

Most of the studies looking into the impact of UICs from an industry perspective focused on how UICs affect business innovation performance (Al-Ashaab et al., 2011; Apa et al., 2021; Fan et al., 2019; Jones & Corral de Zubielqui, 2017). Even though the majority found a positive impact on innovation (Fan et al., 2019; Zhang et al., 2019), when the unit of analysis is small and medium-sized enterprises, empirical evidence showed that formal UICs do not necessarily induce positive innovation performance without the presence of informal relationships (Apa et al., 2021). Similarly, UICs, when consisting of companies with low absorption capacity, do not have a significant impact on innovation (Vega-Jurado et al., 2020). This heterogeneity in results is generally related to the type of company, type of relationship, partner, and absorptive capacity. Therefore, each of these factors should be considered with caution in the impact measurement process (Acebo et al., 2021).

Result heterogeneity is also evident in the measurement of the academic perspective. The literature reveals some concern about how commercialization objectives in the industry undermine scientific production (Perkmann & Walsh, 2009). A recent explanation relates this effect to the attention theory of firms, i.e., high levels of collaboration generate many ideas and low publication rates (Banal-Estañol et al., 2015). Other studies consider that academic institutions can indeed experience intellectual benefits (De Fuentes & Dutrenit, 2012), but with diminishing returns as the time spent in the industry increases (Banal-Estañol et al., 2015), or when an academic is involved in several collaborative projects (Di Maria et al., 2019).

In contrast to the previous argument, Bikard et al. (2019) state that the low scientific productivity is explained by the fact that the universities decide to collaborate in projects more oriented to commercial results than to scientific outputs, or UIC participants do not otherwise apply the advantages of specialization, i.e., delegating responsibilities according to the specialty of each participant. Thus, if the commercial activity is carried out by industry members and the scientific production by academics, the results would benefit all stakeholders (Bikard et al., 2019).

Although only a fraction of collaborative research results in co-authorship, sectors such as electronics, pharmaceuticals, and biotechnology tend to produce more scientific production(Tijssen, 2012). There will always be a risk that some academics may shift the focus of their concern, i.e., they will be more concerned with the commercial outcome of their product than with the content of their scientific output (Bornmann, 2017).

In any case, the trade-off between participating in collaborative projects with industry and the decrease in academic productivity with a high impact factor implies an opportunity cost that is worth thinking about when significant socioeconomic impacts are generated (Di Maria et al., 2019). Certain scholars emphasize the important role of incentives to engage academics in collaborative efforts with the industry (Puerta-Sierra et al., 2021; Skute et al., 2019). Other authors acknowledge the importance of delving deeper into the understanding of the factors that encourage academics to engage with the industry, aiming to enhance the effectiveness of policies promoting such collaborations (Abramo & D’Angelo, 2022).

Once it is acknowledged that long-term collaborations with industry has limitations due to diminishing returns in scientific production (Garcia et al., 2020), several studies have proposed measures, such as the institutionalization of interdisciplinary UIC. For instance, creating incentives distinct from common requirements for scientific production by adopting criteria related to social, human, and financial spheres (Galán-Muros et al., 2017). Accordingly, there are proposals to reformulate the assessment of scientific activity in a more equitable manner and realign certain policies, often conflicting due to their encouragement of high-impact scientific production and simultaneous encouragement of academic engagement in public–private collaborations (Abramo & D’Angelo, 2022).

From a social perspective, UICs have been widely recognized as a significant source of skills and specialized knowledge, playing a crucial role as intellectual capital that drives job creation and wealth generation (Guerrero et al., 2021). Moreover, when social needs are met through responsible innovation generated by UICs, regional economies grow and education systems improve (Acebo et al., 2021; Audretsch et al., 2019). Recent studies state that society imposes certain demands on universities, which are related to social aspects such as poverty relief, inequality reduction, and an enhanced quality of life for individuals. Thus, UICs, as innovation systems, provide a viable means to address each of these challenges (Puerta-Sierra et al., 2021).

One aspect worth mentioning is the relationship between societal impact and the type of country where the UIC is based. The literature points out that in developed countries UICs are driven by commercial, economic and reputational factors, while in developing countries UICs result from the very needs and challenges these countries face, which may explain why collaborations in developing countries generate higher societal impact (Roncancio-Marin et al., 2022).

After summarizing the analysis of the 92 selected articles, 25 UIC impacts were pointed out. Table 1 introduces the impacts and their descriptions. Additionally, we have proposed five impact categories: 'type', 'agent', 'time', 'incidence' and 'nature'. The 'type' category is directly related to the affected area. Consequently, the 25 impacts identified have been classified into six types: intellectual, economic, technological, environmental, social and strategic, as evidenced in Table 1. The different types of impacts can be defined as follows:

Table 1 UIC impacts
Full size table

  • Intellectual: Impacts that directly affect the academic and the industrial communities. These are closely linked to scientific production, the resolution of industrial issues, and opportunities to enhance the capabilities and experience of human capital (De Fuentes & Dutrenit, 2012), alongside the improvement of the educational system and learning processes (Zavale & Schneijderberg, 2021);

  • Economic: Impacts related to the financial outcomes that arise from the development of new ventures, the commercialization of innovative products, and the optimization of resources. This type of impact can emerge after a series of interactions over time between the university and businesses (Azagra-Caro et al., 2017) and is often linked to the increase in anticipated capital and wealth generation (Audretsch et al., 2019). Economic impacts in the field of UIC have been extensively analyzed in the literature (Puerta-Sierra et al., 2021; Roncancio-Marin et al., 2022; Yeo, 2018) and from the university perspective, the economic impact becomes evident as the presence of financial resources allocated to research increases (De Fuentes & Dutrenit, 2012);

  • Technological: Impacts that arise from the implementation of new technologies or innovative concepts in collaborative projects between academic institutions and industry. These innovations can lead to both positive and negative consequences in various spheres, such as productivity, quality of life, job creation, and the environment, among others. Consequently, the degree of efficiency with which these new ideas are transformed into marketable products and services becomes a crucial element in fostering the creation of new innovation mechanisms (Audretsch et al., 2019).

  • Environmental: Impacts related to the outcomes, whether positive or negative, arising from the activities conducted within the collaborative project that directly or indirectly influence the environment (Zhang et al., 2022). Among the environmental impacts documented in the analyzed literature, noteworthy examples include the mitigation of pollutants (Al-Ashaab et al., 2011; Albats et al., 2018) and the advancement of practices that promote the use of recyclable materials (Al-Ashaab et al., 2011).

  • Social: Impacts across several groups of society, encompassing crucial domains such as employment generation (Apa et al., 2021; Wong & Singh, 2013), quality of life enhancement (Zavale & Schneijderberg, 2021), and entrepreneurial endeavors aimed at meeting community demands (Acebo et al., 2021; Audretsch et al., 2019; Roncancio-Marin et al., 2022).

  • Strategic: Impacts that directly affect the image of an organization in its environment, particularly their reputation (Crespo & Dridi, 2007; De Fuentes & Dutrenit, 2012; Galan-Muros & Davey, 2019). Namely, strategic competitiveness (Acebo et al., 2021; Galan-Muros & Davey, 2019), and the organization’s ability to foster future collaborations (Al-Ashaab et al., 2011; De Silva et al., 2021; Zavale & Schneijderberg, 2021).

The second category, known as ‘agent’, represents the community directly or indirectly affected (Table 2). In literature, it is possible to observe that an impact can affect more than one agent, thus demonstrating that the knowledge generated within the UIC can have broader impacts on many areas, namely, industries, universities and society (Galán-Muros et al., 2017).

Table 2 UIC impacts by stakeholders
Full size table

As for the third category, 'time', both the short- and long-term are considered. In this regard, the literature agrees that impacts on the collaborative lifecycle can occur in both timeframes (Maietta, 2015; Siemieniako et al., 2021; Yeo, 2018).

The fourth category, 'incidence', relates to the nature of the impact, whether direct or indirect (Maietta, 2015). In this research, we define direct impacts as those generated from the activities carried out within the collaborative lifecycle. These impacts are observable in the short-term and primarily affect the communities directly involved in the project. Conversely, indirect impacts are those that were not foreseen and can be considered an extension of the effects produced by the activities carried out under the collaborative lifecycle. These impacts occur in the medium and long term, affecting not only stakeholders, but also other social groups.

The fifth category, 'nature', is related to the tangible or intangible features of the impact (Bellini et al., 2019; Fernandes & O’Sullivan, 2021; Perkmann & Walsh, 2009). In this document, we establish a relationship between the concepts of tangibility and intangibility and the level of complexity associated with measuring their impact.

In other words, when there is an exact measure of the impact, it is considered measurable. Conversely, if the nature of the impact is intangible, it does not imply that it is impossible to measure; rather, it requires more sophisticated assessment approaches and tools. Figure 2 organizes the 25 UIC impacts under five impact categories.

Fig. 2
figure 2

UIC impact categories

Full size image

Finally, Fig. 3 illustrates the evolution over time of the six types of impact in literature. To achieve this, we have divided our period of analysis (2000–2022) into four specific periods (2000–2004, 2005–2009, 2010–2014, and 2015–2022) and counted the number of articles that address each impact type, thus providing a visual representation of how the analysis of each impact has evolved over time in the context of UICs.

Fig. 3
figure 3

Temporal Evolution of the Six Types of Impact

Full size image

Figure 3 shows a consistent and growing interest by economic impacts since the period 2000–2004. Similarly, intellectual impacts gained in prominence in the literature from the second period onward, alongside the economic impacts. This initial trend substantiates our findings regarding how academic and industrial perspectives have been scrutinized in the literature on UICs. Consequently, it underscores the need to analyze more comprehensive impacts that transcend both academic and industrial realms.

The significantly deeper analysis of social impacts in the last period analyzed suggests that the literature is responding to the call to address social needs. A plausible explanation for this trend could be linked to the fact that social impact is a great concern of policymakers and professionals involved in the commercialization of science (Fini et al., 2018).

Concerning the strategic and technological impacts, although not growing in the number of articles at the same pace as the others, these impacts still catch the interest of literature.

Regarding the environmental impacts resulting from UICs, few studies have delved into this subject. This result emphasizes the historical lack of additional research on environmental indicators, as indicated by Karatzoglou (2013) in his literature review on the university's role in sustainable development. After looking into 123 articles, Karatzoglou identified only three which focused exclusively on measurement systems, highlighting the urgent need for more in-depth studies in this field.

4.2 Challenges in measuring UIC impacts

The reflexive thematic analysis allowed us to identify four recurring methodological challenges in measuring the impact of UICs. The first challenge concerns the 'multidimensional nature of the impact', whether tangible or intangible (Bellini et al., 2019; Soh & Subramanian, 2014), due to the uncertainty of the impact appearing in the long-term or short-term (Maietta, 2015; Yeo, 2018), to its direct or indirect impact (Perkmann & Walsh, 2009) or to its positive or negative effect in a given area (Fini et al., 2018). All of these aspects increase the complexity of measuring any impact that takes time to materialize (Perkmann & Walsh, 2009).

The second challenge identified was 'causal attribution of effect'. A clear example thereof is the difficulty in establishing whether an increase in sales is the result of the UIC, since there may be other factors that influence its performance (Perkmann et al., 2011). Although some studies suggest that UICs positively influence the sales of innovative products (Arvanitis et al., 2008), or improve a company's market value (Crespo & Dridi, 2007), there is no clear understanding of the results derived from collaboration (Galan-Muros & Davey, 2019). So there is the risk of alternative causes explaining such effects (Fini et al., 2018). Therefore, knowing to what extent the collaboration was useful for achieving innovative results, or even knowing what would have happened if the collaboration had not taken place are still counterfactual issues to be considered (Lööf & Broström, 2008).

Formal collaboration in R&D encompasses a wide range of cooperative research and knowledge transfer activities, involving continuous interactions between stakeholders (Wong & Singh, 2013). When these continuous interactions are of high professional value, they tend to ensure the existence of societal impacts (Bornmann, 2017) However, there is a challenge related to the 'identification of impacts', both perceived and expected. Identifying perceived impacts requires addressing significant differences in individual and institutional perceptions that are subjectively correlated with affective evaluations. The perceived benefits, particularly those related to future collaborations, are thus positive (De Silva et al., 2021). Moreover, identifying expected impacts is a challenging exercise due to the risks and uncertainties inherent to UICs (Fernandes & O’Sullivan, 2021).

To measure the impact of research on society, it is often more convenient to compile data at the institutional level than at the individual level, since institutional data are more easily identified (Bornmann, 2017). However, more than identifying data, another frequent difficulty in measuring the impacts of UICs is having sufficient and appropriate information, especially when it comes to micro-level data (Yeo, 2018). Thus, we have identified a fourth challenge, which we have called 'data limitations', explained by some authors as the absence of a culture of periodic recording of information by organizations (Penfield et al., 2014).

The literature related to UICs showed that 'data limitations' can occur for different reasons, such as low stakeholder participation in surveys, as evidenced by a low response rate or a significant number of contradictory answers (Arvanitis et al., 2008). 'Data limitations' can also occur because of the short period in which the information remains available or because the available data do not reveal the specific realities of the context under analysis (Zavale & Schneijderberg, 2021). Particularly, in the context of co-financed R&D projects, data limitations are influenced by geographical, economic, scientific and cultural factors that often constrain the integrity and validity of the data (Tijssen, 2012). Whatever the reason, this problem may limit the scope of the study by having to exclude part of a sample, for example small companies, from a study (Maietta, 2015). In contrast, when data are recorded extensively in organizations, such impact is more likely to reveal itself (Yeo, 2018), which facilitates the study of longitudinal phenomena (Arvanitis et al., 2008).

4.3 Strategies used to measure UIC impacts

Our literature review allowed us to identify some strategies for UIC impact measurement. Given the 'multidimensional nature of impact' challenge and the tendency of traditional measurement methods to focus on the last stage of the collaboration lifecycle, i.e. the output phase of patents, licenses and joint publications, some authors have proposed the implementation of 'continuous monitoring throughout the UIC lifecycle', i.e., inputs, activities in process, outputs, and impacts (Albats et al., 2018). 'Continuous monitoring throughout the UIC lifecycle', accompanied by the 'interactive participation of stakeholders', is expected to foster the evaluation and balanced selection of the most appropriate indicators to measure impact, and to point out the appropriate direction for future collaborations (Albats et al., 2018).

A second strategy used to measure UIC impacts is the 'combination of data collection tools', namely by the use of primary data collection tools, such as case studies, interviews and narratives (Al-Ashaab et al., 2011; Borah et al., 2021; Morandi, 2013; Perkmann & Walsh, 2009) and the use of secondary databases (Scandura, 2016; Vega-Jurado et al., 2020; Wong & Singh, 2013; Zhang et al., 2019). We believe that the combined use of these data collection tools (e.g., Soh & Subramanian, 2014; Wirsich et al., 2016; Wong & Singh, 2013) can contribute to a more complete picture of the UIC context and capture the different views of academic, industry and society stakeholders (Albats et al., 2018), becoming an important strategy for dealing with 'data limitations' and facilitating the 'identification of impacts', both perceived and expected.

The literature also suggests the implementation of 'benefit management systems' that allow for a more precise identification of expected benefits and the allocation of responsibilities among participants in the UIC (Fernandes & O’Sullivan, 2021). A 'benefit management system' involves a set of interactive activities among UIC members, aimed at identifying, reviewing, executing and projecting future actions (Fernandes & O’Sullivan, 2021). Considering that benefits have a positive connotation, while the definition of impact used in this paper implies positive and negative effects derived from the UICs, henceforth we will use the term 'Impact Management System'. Thus, the implementation of this system could contribute to building an ideal context involving interactions between academics and entrepreneurs, which are fundamental in the creation of common benefits (Galan-Muros & Davey, 2019). However, it is an exogenous strategy, beyond the scope of the UIC impact evaluator, and its implementation depends on the UIC organizations involved.

Fini et al. (2018) guide impact evaluators to go beyond traditional measures and use 'digital and technological tools' to build more efficient databases. For instance, web-based metrics (altmetrics) can help map the broader impact of research by connecting interactions on social networks between scientific production and various groups, such as public policymakers (Bornmann, 2017). Furthermore, the 'use of multidisciplinary approaches' that consider ethical and moral issues is also considered relevant when the commercialization of innovation generates positive and negative impacts simultaneously. For example, important technological innovation may generate negative environmental impacts related to moral and ethical issues that must factored into impact measurement (Fini et al., 2018).

Impact causality in the field of UIC has been addressed with 'parametric/non-parametric and qualitative methods', such as estimators (e.g. instrumental variables) and nonparametric estimators (e.g. matching estimators) (Lööf & Broström, 2008). The latter impose a condition of independence to determine whether the impacts would be possible in the absence of collaboration (Scandura, 2016), and empirically addressed counter factuality, using regression models and propensity score matching for comparative analyses between groups of companies participating in the collaboration and a non-collaborative control group, aimed at estimating the impact of the UIC. Another strategy identified was qualitative approaches using questionnaires to obtain participants' views on what would have happened in the absence of the collaboration (Wooding et al., 2007).

5 Discussion

This research builds on knowledge of UICs by delivering a macro-level perspective on measuring broader impacts of UICs for which there is limited understanding (Bornmann, 2013; Di Maria et al., 2019; Jones & Corral de Zubielqui, 2017; Nugent et al., 2022). More specifically, the contribution of this paper is twofold. Firstly, we propose a framework that outlines the process of measuring the impacts of UICs, integrating impact categories, the challenges associated with their measurement and strategies to overcome them.

The framework introduces the UICs as a cyclical process through which useful knowledge of high social impact can be produced. This cycle is basically divided into four phases, inputs, in-process activities, outputs and outcomes (Galan-Muros & Davey, 2019; Perkmann & Walsh, 2009). In each of these phases there is a degree of interactivity among the participants and with it the probability of generating direct or indirect impacts at the individual or community level (Bornmann, 2017). These impacted groups or communities may belong to industry, academia, and other social groups, such as funding institutions or those responsible for public policies, etc. Consequently, the identification of impact and the diverse groups affected requires caution and a comprehensive understanding of the multidimensional nature of impact to avoid unidirectional effects or underestimation of the existence of bidirectional knowledge flows (Verre et al., 2021).

The framework proposes a categorization of the impacts identified in the literature. Six types of impacts related to intellectual, economic, technological, environmental, social, and strategic areas (see Table 2 and Fig. 2) are identified. It is worth mentioning that the impacts belonging to the strategic category, such as reputation, competitiveness, and future collaborations are clearly mentioned and recognized in the literature. However, they are not commonly included in empirical studies of UIC impact measurement. Therefore, since there is a broad consensus on the existence of these impacts, strategies must be devised for their inclusion in future impact measurement methodologies.

The framework also shows four types of challenges related to impact measurement, namely, 'multidimensional nature of impact', 'causal attribution of effect', 'identification of impact' and 'data limitations'. These challenges explain the scarcity of systematic studies attempting to measure the impacts of UICs, complementing the findings in the literature on conceptual issues in impact types (Bornmann, 2013; Donovan, 2011).

Secondly, this research contributes to drawing clear connections between the challenges of impact measurement and the strategies to overcome these challenges. We contend that identifying these challenges not only enhances the likelihood of advancing the measurement of UIC impacts but also contributes partially to strengthening the connection among the thematic elements comprising the UIC ecosystem (Galan-Muros & Davey, 2019). In Fig. 4, we use the colors green, lilac and yellow to visualize these connections.

Fig. 4
figure 4

UIC impact measurement framework

Full size image

Thus, the challenge of the 'multidimensional nature of impact' can be addressed through continuous monitoring across the lifecycle of the collaboration and interactive participation of project stakeholders. The challenge of ‘causal attribution of effect’ has been addressed in the literature mainly through qualitative methods that need to be combined with quantitative tools to build more robust methods of measurement (Bornmann, 2013). However, theory points to new paths in future research, namely by using 'multidisciplinary approaches', 'parametric/non-parametric and qualitative methods' and 'Digital and technological tools' for collecting and managing information, and to the connection of different areas of knowledge to allow for a broader analysis of the impact (Fini et al., 2018). Nevertheless, no empirical evidence applying these strategies was found in the analyzed articles.

The literature also shows that challenges related to 'data limitations' and 'impact identification' can be addressed through the 'combination of data collection tools', which was a common strategy among the analyzed studies (e.g., Soh & Subramanian, 2014; Wirsich et al., 2016; Wong & Singh, 2013). Many of them went beyond a particular source and used various tools to gather information, such as narratives, interviews, focus groups, and existing secondary databases.

Another strategy mentioned in the literature was the implementation of an 'Impact management system' (Fernandes & O’Sullivan, 2021), which helps to identify the expected impacts. We emphasize this last strategy, because it is an external mechanism that falls outside the control of the impact evaluator. We believe that UICs that manage to adopt this type of system could develop the capacity to provide more complete information on the changes experienced during the project, which would benefit future empirical studies that seek to analyze its impacts.

The set of seven strategies discussed, shown in the last layer of Fig. 4, contributes partially to each of the four identified challenges of measuring UIC impacts. While all these strategies converge towards a common goal, which is the continuous monitoring of impact through information captured from many sources, the identified challenges can be intricate and require multiple and complementary strategies. Nevertheless, the linkages between strategies and challenges discussed in this paper, while not unique, can serve as a basis for the process of measuring impact in the UIC context.

Finally, it is worth mentioning that the potential 25 impacts resulting from UICs can significantly vary in terms of their impact degree. For example, (I.7) New business opportunities (e.g., creation of spin-offs and start-ups), (I.9) New and improved products and services, (I.14) Development of new technologies, (I.19) Promotion of regional economic and social development, and (I.20) creation of jobs, may have a significant potential. However, attaining a significant degree of these impacts requires achieving a deep change, capable of expansion at local, regional, national, and international levels, and showing long-term sustainability (Scoble et al., 2010).

The challenge of attaining a more significant level of impact may be linked to various inhibiting factors, including the lack of shared objectives among universities, science, and businesses (Issabekov et al., 2022). The absence of these common objectives can significantly affect the level of commitment from the involved parties, which is a key factor in driving the innovative performance of companies collaborating with universities (Marra et al., 2022). Additionally, as highlighted in the literature, information asymmetry increases uncertainty levels between the university and industry (Xiaojuan & Hongda, 2021). These inhibiting factors require sustainable strategies to maintain common interests over time (de Freitas et al., 2014), and facilitate the occurrence of more comprehensive impacts.

6 Conclusions and future avenues of research

University-industry collaboration in R&D projects (UICs) are characterized by their ability to impact individuals or groups in a society. However, measuring and demonstrating such impacts is a complex task that requires thorough analysis. Resorting to systematic literature review, this study identifies different types of impacts in the context of UICs, as well as the challenges of measuring such impacts and the strategies that can be used to overcome them (see Fig. 4). We propose a categorization of the impacts of UICs based on the 'type' (intellectual, economic, technological, environmental, social and strategic), the ‘agent' (industry, academy and society), the 'time' the impacts take place (short- or long-term), the 'incidence' (direct or indirect) and the 'nature' (tangible or intangible).

The categorization of impacts by 'agent' emphasizes the need to conduct empirical studies that consider the viewpoint of each stakeholder, as their interactive participation facilitates the identification of the impact, the specific group and area that may be affected by the activities carried out at the UIC. In this regard, it is crucial to acknowledge that, apart from the university and industry, society itself plays a pivotal role, represented by various academics, industrials, regions and other communities that are part of society.

We believe that the remaining three UIC impact categories (i.e., 'time', 'incidence' and 'nature') reflect the 'multidimensional nature of impact', which represents an challenge inherent in its measurement process. Likewise, the literature review allowed us to identify additional challenges related to 'causal attribution of the effect', 'data limitations' and 'impact identification'. These challenges further muddle the measurement of impact and explain the scarcity of studies that have attempted to analyze the former.

Various methodological strategies were identified in the literature to address the aforementioned challenges. However, some of these strategies, such as 'continuous monitoring throughout the UIC lifecycle', the use of 'multidisciplinary approaches', or of 'digital and technological tools' were not empirically applied in the studies analyzed. Nevertheless, there are exceptions worthy of note, such as the ‘combination of data collection methods', i.e. quantitative and qualitative methods, as well as the integration of databases, which have been frequently used in impact measurement studies, in view of obtaining a more comprehensive understanding of organizational-level impacts (Al-Ashaab et al., 2011; Borah et al., 2021; Morandi, 2013; Perkmann & Walsh, 2009).

This research represents a significant theoretical advancement in our understanding of UICs by skillfully integrating diverse strategies aimed at tackling the intricate challenges associated with measuring their impacts. By summarizing these strategies, we are aiming at the development of forthcoming methodologies capable of effectively encompassing the various impacts mentioned in literature, which are currently not suitably addressed in measurement processes due to their intangible and complex nature.

The current study’s findings have some limitations. Although a systematic review process was employed, as pointed out by Xia et al. (2018), literature reviews are never exhaustive. Therefore, in this process some articles may have been left out of the analysis. Possible exclusions may be the result of various factors, such as the choice of keywords, the string used, the scope of the search, or methodological gaps that could potentially be identified by other researchers. These issues present opportunities for future research or extensions of the current study. Besides, in the literature analysis process, specifically in qualitative and reflexive thematic analysis, cognitive bias cannot be entirely eliminated. Thus, while the results obtained offer suggestions for the measurement of UIC impacts, they are not confined to the framework proposed herein.

For future investigations is critical to focus on the perspectives of all parties involved in the collaboration to comprehend the perceived and anticipated impacts of these UICs. This involves employing diverse methods of information gathering to ensure that socially recognized impacts theoretically acknowledged are adequately included in the measurement process.

Future empirical research is also needed to delve into the knowledge of new strategies that can be implemented to overcome the challenges of measuring the broader impacts of UICs. For instance, field studies addressing the perspective and experience of the actors directly involved in collaborative project management can deliver valuable inputs. Additionally, promoting the use of multidisciplinary approaches, the application of digital and technological tools, and the combination of quantitative and qualitative methods can enrich the assessment of impacts and provide a more comprehensive understanding of the effects generated.

Future research could also further explore the inhibiting factors influencing the impacts stemming from UICs and the strategies that can be implemented to attain a more comprehensive level of impact. We recognize that there are impacts with high potential, such as: new business opportunities (e.g., creation of spin-offs and start-ups), new and improved products and services, development of new technologies, promotion of regional economic and social development, and creation of jobs.

However, strategies must be implemented to overcome inhibiting factors of these impacts. For instance, instituting regular innovation meetings on-site, involving multidisciplinary teams within the company and university, can serve as an effective approach to overcome such inhibiting factors, providing a platform to explore and assess the impacts resulting from the collaborations (Penfield et al., 2014). Such meetings have the potential to stimulate the development of new products, processes, and technologies that promote regional economic development, as well as to increase the level of trust and the likelihood of future collaborations (Fernandes & O’Sullivan, 2021).

Furthermore, not all economic development contributes to regional innovation (Cui & Li, 2022). Therefore, there is a need for policies that facilitate and incentivize greater exploration of collaboration results within the region itself. This strategy can play a significant role in promoting sustainable economic and social development while also fostering the creation of new businesses and increasing local employment.

Finally, the majority of the impacts of UICs recognized and identified in the literature are positive in nature. While few studies acknowledge the existence of negative impacts stemming from UICs, these are not explicitly identified, except the impact related to potential reduced scientific productivity among academics. Therefore, additional empirical studies are needed to explore other specific negative impacts that may arise from collaborations with industry. Additionally, it will be worthy to analyze potential political or organizational hindrances to accessing information related to the negative impacts that could result from co-funded collaborations between industry and university.