Background

Increasingly, researchers are requested to explicitly address collaboration in their research proposals (Braun & Griessler, 2018; European Commission, 2018; Perkmann et al., 2013; Rolfe et al., 2018). Reasons underpinning these policy-driven requests are the expectations that societal challenges can be met through integrated research approaches (Graham et al., 2006, 2019; Macq et al., 2020; Powell et al., 2018).

Collaborative research approaches that support actors outside universities to have an essential role in research are increasingly important to reveal the complexity inherent in global societal challenges, thus addressing the relevance aspect of research (Bammer, 2019). In research targeting challenges in the welfare systems, collaboration is recommended as a means to strengthen societal impact (see e.g., AGE Platform Europe, 2014; Greenhalgh et al., 2019) and collaboration is expected to generate research that is relevant, timely, useful and used (Graham et al., 2019). At the same time, the idea is that actors outside the university strengthen and develop their ability to translate research outputs into practice. Emanating from a democratic rights-based perspective, ideas of empowerment often constitute the origin of motivations for collaboration with citizens, making efforts to change positions of strength related to influence over the research process from researchers to partners outside the university (Bammer, 2019; Braun & Griessler, 2018; Mockford et al., 2012; Stilgoe et al., 2014). Others advocate for collaboration as a means of systems thinking, strengthening the quality of research through an increase in researchers’ understanding of actors outside the university, their expectations, issues, contexts, and challenges as well as their opportunity for and ability to be involved in research processes (Fritz et al., 2019; Graham et al., 2019). However, little is known about how such requirements and ambitions are played out in guidelines for applicants, proposals, peer reviews, and funding decisions.

There is a firm belief that increased collaboration with actors outside the university can contribute to the creation of new knowledge and innovations as well as positive societal developments. This entails a recognition that research-based knowledge exists along with other forms of knowledge such as local, lived, or applied knowledge and competes with other priorities (Durose et al., 2018; Powell et al., 2018). In some disciplines and research fields, this includes the view that research is one component among others in complex social and political processes, making it challenging to evaluate the direct effects and impacts of collaboration in research (Molas-Gallart & Tang, 2011).

Study context

The Swedish Research Council for Health, Working Life and Welfare (Forte) is a government agency under the Swedish Ministry of Health and Social Affairs. In their input to the Swedish Government’s research policy bill (2016) Collaborating for knowledge—for society’s challenges and strengthened competitiveness, Forte emphasized the importance of collaboration and utilization to promote societally relevant research, which was later spelled out in their calls for research proposals (Forte, 2015, 2019).

In 2016, with changes in their application forms and assessment criteria, Forte wanted to stimulate researchers to reflect on the significance of collaboration, societal relevance, and knowledge dissemination and integrate such perspectives and ambitions in their research proposals. Because the contexts for collaboration differ among the diversity of disciplines and research fields addressed by Forte, the instructions to the applicants were open (Table 1). In this context, collaboration was described as collaboration or cooperation with relevant actors, stakeholders, or individuals outside the university, but was not qualified in more detail by Forte. They were thus intended to enable applicants to link their descriptions to specific interests, approaches, networks, resources, and challenges.

Table 1 Subheading and instruction for the proposal text field that constituted the basis for analysis
Full size table

Overlapping collaboration concepts, paradigms, and goals

There are a considerable number of terms and concepts that relate to collaboration in research. For example, Participatory Research, Transdisciplinary Research, Patient and Public Involvement, Responsible Research, and Innovation and Engagement Science.

Collaboration is an important facet of the knowledge-to-action (KTA) field (Graham et al., 2006). Various terms are used, such as Integrated Knowledge Translation, Knowledge Transfer, Knowledge Exchange, Knowledge Mobilization, Mode-2 Research, and Participatory Research (Graham et al., 2006; Harder et al., 2013; Nguyen et al., 2020). The various terms are related to continuously evolving research policies, which in turn are shaped by factors such as broad societal crises, specific epistemic communities, dynamics within institutions, and individual commitments of key actors (Macq et al., 2020). Due to multiple pressures and frustrations, such concepts are developed in parallel, with few points of intersection (Powell et al., 2018). However, there are similarities such as the partnership approach, focus on core values, principles, and collaborative processes, collaborative research orientation, and the need for extensive time and financial investments (Hessels & van Lente, 2008; Nguyen et al., 2020). Concepts differ in the views on the various actors involved, the purpose, motivation, nature of knowledge, use processes and outcomes, boundaries of the field, and historical underpinnings (Graham et al., 2006; Nguyen et al, 2020). The meanings and scopes of concepts carry multiple interpretations in various areas of knowledge (Harder et al., 2013; Straus et al., 2009). Terms such as co-creation, co-production, co-design, participation, involvement, and engagement have different meanings in different disciplinary and national contexts (Locock & Boaz, 2019). The concepts are more or less flexible to adapt to various contexts (Macq et al., 2020). Terminology indicating one-way communication is increasingly questioned, while terminology that emphasizes dialogue and active translation of new knowledge is in favor (Bjursell et al., 2016; Locock & Boaz, 2019; WHO, 2012). Studies of an interactive and participatory research communication signal that scholars are ready for this shift (MacGregor & Cooper, 2020).

Participatory Research can be seen as an umbrella term for approaches (e.g. action research, community-based participatory research, and participatory action research) that share a core philosophy of working together with those ultimately affected by the research, questioning hierarchies, mutual learning, and effecting social change, particularly to benefit marginalized groups (Nguyen et al., 2020). Transdisciplinary Research (Thompson Klein, 1990) implies interdisciplinarity, involving collaboration among researchers with different backgrounds with similar interests, extending to the co-production of knowledge with actors outside the university (Lawrence, 2015). Patient and Public Involvement (PPI) plays an integrated role in engaging service users, patients, consumers, or caregivers in shaping health and social care services (Mockford et al., 2012; Staniszewska et al., 2018). PPI emphasizes research with rather than for the public. The goal is not only to maximize societal impact but also to take the new problems or ethical dilemmas that collaboration can entail into account. This is denoted by the emerging research policy framework Responsible Research and Innovation (RRI) (Smallman, 2018), emphasizing that research and innovation must engage with the public to foster inclusive and sustainable research and innovation (Braun & Griessler, 2018). Adding yet another term, Engagement Science refers to the inclusion of non-traditional actors as members of research teams throughout research processes (Cope et al., 2019), investigating collaboration methods and practices, and developing evidence-based approaches (Dungan et al., 2019).

In research policy as well as in research proposals and published results, terms representing this multitude of concepts are often used in parallel and sometimes overlapping. Overall, generally accepted definitions are lacking, and the concepts are ranging from passive communication with actors outside the university to full partnerships with equal partners (Graham et al., 2019). No matter what concepts are used, researchers and actors outside the university are both producers and consumers of knowledge, and the common aspiration is to better connect research with society to address societal challenges.

Collaboration as a criterion for funding

Many researchers engage a wide range of actors outside the university and the forms of collaboration are varied (Abreu et al., 2009). According to Bammer (2019), not engaging actors outside the university and limiting co-creation limit the ability to make complexity evident and to act effectively on complex societal challenges. However, collaboration as an assessment criterion for funding challenges several of the values ​​and assumptions that researchers hold (Gradinger et al., 2013) and not all are advocates for collaboration with actors outside the university. For some researchers, collaboration may not be necessary to strengthen research quality and fulfill their wider university role (Abreu et al., 2009) and some do not perceive collaboration activities as proper academic work (Molas-Gallart & Tang, 2011). For example, in a national study, one of five Swedish researchers stated that actors outside the university should have no influence whatsoever on the research process (Bohlin & Bergman, 2019). Another example is a recent study with responses from Swedish ageing and health researchers, showing that there does not appear to be a consensus on whether or not actors outside the university should be involved in research (Kylén et al., forthcoming). Researchers may perceive that the traditional curiosity-driven approach is under threat (McLean et al., 2018). In relational approaches to research, social interaction and negotiation among actors are essential and dependent on opinions, judgements, values, dynamics, and power structures (Powell et al., 2018). According to Bandola-Gill (2018), institutional and cultural duality is a challenge for researchers who experience contradictory expectations and guidance by conflicting incentive systems. Paylor and McKevitt (2019) suggested that collaboration as an assessment criterion has three consequences: (1) different demands and logic for researches; (2) acts of recalcitrance and impression management to secure funding; (3) destabilization of researchers’ professional identity. Advocates of collaboration claim that the benefit, is the research of high quality that is useful for society. But, collaboration may also hinder efforts or give negative effects (Czarnitzki & Toole, 2010) due to lack of time (Abreu et al., 2009) and resources, differing time scales, conflicts of interest and commitment, and barriers caused by intellectual property rights issues (Gulbrandsen & Smedby, 2005). According to Powell et al. (2018), even research funders, researchers, and intermediaries do not fully uphold the principle that research-practice gaps are most effectively bridged when other forms of knowledge are acknowledged and combined with research-based knowledge. Paradoxically the ongoing shift in the KTA field has made collaboration more complex and uncertain and has made the role of the researcher more diverse and demanding (Powell et al., 2018).

Overall, how this ongoing and complex development is spelled out and operationalized in the context of research funding is largely unknown and the effects on, changes for, or benefits to society remain to be demonstrated. Despite the increased importance of the collaboration criterion to legitimize research, the knowledge about collaboration efforts in research is insufficient, and little is known about the actual impact, challenges, and costs (Durose et al., 2018; Mockford et al., 2012; Perkmann et al., 2013; Shippee et al., 2013).

Study aim

Aspiring to contribute to research policy development, the overall ambition of the present study was to follow up and analyze how applicants interpreted and adopted Forte’s new requirements, how the criteria announced were assessed and what weight they were given by the reviewers. The specific aim was to scrutinize how the collaboration criterion was operationalized in research proposals and related assessments. The following research questions guided analyses informed by a synthesis of current concepts and theoretical underpinnings:

(1) How did applicants respond to the request for collaboration spelled out by the research founder?

(2) How did the review panels evaluate these responses?

Methods

We used qualitative manifest content analyses (Elo & Kyngäs, 2008) to identify to what extent overarching trends were revealed in the material, and quantitative analyses to describe the content, interpretations, and adaptions related to the collaboration criterion.

Text units for analysis

The material consisted of 381 proposals for project and program grants in the research fields of health, working life, and welfare during 2016, of which 109 (29%) were granted (see Table 2). The proposals for 3-year project grants originated from Forte’s annual open call. The proposals for 3 + 3-year program grants originated from a strategic research field call with the purpose to support the long-term development of research environments within five broad program initiatives: Ageing, Demography, and Health (Ageing); Transformations and Challenges concerning Working Life and Labour Market (WorkLi); Alcohol, Narcotics, Doping, Tobacco, and Gambling (ANDTG); Migration and Integration (Migrat); and Equal Living Conditions (EquaLi). Moreover, the material consisted of 15 review panels’ assessments of the 381 proposals, serving as the basis of the final funding decisions subsequently made by the Board of Forte.

Table 2 Text units for analysis (N = 381 proposals to Forte during 2016)
Full size table

The proposals were registered in the Prisma application and review system. In this online portal, funding agencies post the instructions for specific calls. For the Forte calls in 2016, there were various text fields to be filled in under the two main headings Program Description (eight subheadings with text fields limited to specified nos. of characters) and Dissemination and Collaboration (three subheadings with text fields limited to specified nos. of characters). For examples, see https://forte.se/en/latest/. The material selected for analysis consisted of the applicants’ descriptions under the subheading Collaboration (see Table 1).

The instructions for the review panels were the same for both calls. The headings in the review assessments were not congruent with the ones used by the applicants. The material from the review panels’ assessments consisted of the comments and evaluations registered under the heading Communication and Cooperation. Two studies based on a selection of 17 granted proposals qualified as ageing research and related assessments have been published elsewhere (Hultqvist, 2021; Hultqvist et al., 2021).

Research applications to Swedish Research Councils are publicly available information to any member of the public upon request. The present study represents research, which does not fall within the scope of the Ethical Review Act in Sweden. However, adhering to proper ethical conduct, to anonymize the data, protect sensitive information and safeguard anonymity, no names, organizations, or otherwise identifiable details are displayed. Only researchers involved in the study had access to the data.

Procedure

We used the NVivo software (Edhlund & McDougall, 2016) to organize the data and aid the analysis. As described in detail below, after an initial deductive approach with the creation of preliminary categories, data imported to NVivo was processed with a manifest content analysis according to Elo and Kyngäs (2008) (see Fig. 1).

Fig. 1
figure 1

Procedure of the manifest content analysis

Full size image

Initial deductive approach

Based on a pragmatic synthesis of relevant and central themes, literature, frameworks, and models for collaboration, we defined preliminary categories for an initial deductive manifest content analysis approach. The text field collaboration was expected to consist of descriptions that potentially could fit into the categories and breadth of diverse actors outside the university (i.e. who; Abreu et al., 2009; Harder et al., 2013; WHO, 2012), forms, and depths of participation (i.e. how; Bammer, 2019; Rolfe et al., 2018), phases and scope of participation in the research process (i.e. when; Shippee et al., 2013), reasons motivating and effects of collaboration (i.e. why; Gradinger et al., 2013; Graham et al., 2019; Perkmann et al., 2013). Given Forte’s areas of responsibility we expected to find a broad range of actors representing categories such as citizens, health care, social services, labor-market actors, industry, public agency, policymaker, and interest organizations (Iwarsson et al., 2019; WHO, 2012) represented in the material. The chosen actor constellation and its breadth have a huge impact on the form and depth of participation.

Fitting this to a proper model for different forms of participation, we identified the research-relevant modified International Association for Public Participation (IAP2) spectrum (Bammer, 2019) including the categories of information, consultation, involvement, collaboration, and empowerment. The modified spectrum is suggested to be used to describe different forms of participation with actors outside the university in research and to reflect on the appropriateness of the different ways to engage with various actors tackling complex problems in diverse contexts (Bammer, 2019). We assumed that collaboration could take place in both preparatory, execution, and translation phases (Shippee et al., 2013).

Inductive analysis approach

As we realized that it was not possible to classify the entire material into the categories defined a priori, in the next step codes were allowed to emerge inductively. The first author worked independently with the data in NVivo and met regularly with the second author for consensus discussions regarding the coding. This led to the extraction of meaning units (Graneheim & Lundman, 2004), that is, words, sentences, or passages with aspects that are related to each other through their context and content, to allow labelling of the emerging codes. Thereafter, iterative recoding of meaning units, clarification, and regrouping of codes in emerging categories and subcategories was carried out.

To clarify and distinguish among the data sources, the following abbreviations were used: p = number of proposals; u = number of meaning units; m = mean. To facilitate the readability of the findings, names of categories, subcategories, and codes were italicized.

Word search in the text field collaboration

To analyze concepts for collaborative research approaches, we searched for words in the text field collaboration using NVivo. In this way, we identified occurrences of words, texts, or phrases selected based on relevant literature, models, and frameworks as well as on what emerged in the material. The following words were searched [Swedish terms in brackets]: Method [metod], methodology [metodologi], model [modell], approach [ansats], participatory, [deltagande, deltagarbaserad] perspective [perspektiv], co-production, co-design, co-creation, user-centred, [användarcentrerad], patient and public involvement, responsible research, interaction, involvement [brukarmedverkan, involvering], integrated [integrerad], action research [aktionsforskning], alliances [allianser] and transdisciplinary [transdisciplinär].

Analysis of the review panels’ assessments

Because many of the review panels’ texts consisted only of an evaluative judgement without any specified content, the category evaluation was created. The classification of the review panels’ assessments with specified content followed the categories, subcategories, and codes that had emerged in the analysis of the descriptions in the proposals.

Member checking and feedback

Implementing member checking (Thomas, 2016), we presented preliminary results of the analysis for discussion at a full-day workshop arranged by Forte. Two representatives from each of the 23 granted research programs were invited; 34 researchers representing all five strategically important research fields and seven employees from Forte participated. This dialogue generated valuable comments and questions that were incorporated into the final analysis. An early draft of a report in Swedish was read and commented on by the two Forte employees. A later version also was commented on by an external reviewer appointed by Forte. We incorporated these comments and suggestions into the final report in Swedish (Jonsson & Iwarsson, 2018), which constituted the basis for the present study.

Findings

Descriptions in the proposal text field collaboration

On average, applicants used 1,746 characters of the maximum 2,500 in the text field collaboration. The text field collaboration generated 1,435 meaning units that were consistent with Forte’s instructions in Prisma (see Table 3). Additionally, 322 meaning units that were not consistent with the instructions were generated.

Table 3 Meaning units from the text field collaboration classified in categories, subcategories, and codes [number of proposals (p); number of meaning units (u)]
Full size table

Forms and actors

The largest subcategory was cooperation with partners outside the university (p = 271) (see Table 3), which consisted of meaning units with terms such as cooperation, collaboration, and partnership where the partners were involved in the planning, undertaking, and/or dissemination of the research. The differentiation to consultation with actors outside the university (p = 120) was that such meaning units comprised terms such as consultation, networking, communication, and connection.

A wide range of actors was described and many of the individual applications contained a variety of enumerated actors. Public authorities were the most common and included organizations such as municipalities and county councils. Examples stated were clinics, schools, health and social services, hospitals, and the authority for employment services. Cooperation with public authorities (p = 186) was described in half of the proposals. One of four had meaning units classified as cooperation with policymakers (p = 90). Those who worked on issues related to policy, such as representatives of government, government agencies, and politicians were classified as policymakers. Other actors were classified into NGOs (e.g., foundations, interest/non-profit organizations) (p = 77); industry (e.g., employers, industry associations, consulting firms) (p = 63); laypeople (e.g., patients, family members) (p = 34); unions (p = 25) and intermediaries or independent research organizations (e.g., advocacy groups, non-governmental research institutes) (p = 20).

Reference groups (p = 85) and international collaboration (p = 75) were mentioned in every fifth proposal (see Table 3) with more, less, or not specified frequent sessions for interaction. The code reference group included meaning units such as this example:

We have put together a carefully selected reference group that will convey annually to review the progress and help facilitate national and international impact of the program. (Rejected program proposal, ANDTG).

One of ten proposals had meaning units classified into double affiliation (p = 40) indicating a bottom-up mode of organizing and undertaking collaboration. They were about co-applicants who were managers, board members, or working part-time at one of the partnering organizations, exemplified by this quotation:

As one of the project members is employed by the … and works with policy making and the translation of scientific findings into practice we have pre-existing expertise within the project group on how to disseminate research findings outside the scientific community. (Granted program proposal, Migrat)

Most proposals with meaning units classified into forms and actors showed similarities between granted and rejected. However, less than one of six with meaning units classified into no collaboration (p = 38) was granted. This code included proposals that lacked descriptions of collaboration with partners outside the university, only indicating one-way communication or stating reasons for why there were no plans for collaboration. Six proposals were granted even though the content was classified as no collaboration, but all proposals with reasons for no collaboration with actors outside the university were rejected. Examples of such reasons were basic research, ongoing collaboration in parallel projects, the severity of patients’ conditions, risk of compromising objectivity, independence, and integrity, and unwanted impact from formal collaboration with the context studied. In contrast, almost half of the proposals with actors classified as intermediaries or independent research organizations and more than one of three proposals with actors classified as policymakers were granted. On the other hand, only one of four proposals with actors classified as laypeople were granted.

Phases in the research process

Few applications included clear descriptions of collaborative processes. Half of the proposals had meaning units classified into the category phases in the research process (p = 179) (see Table 3). These were classified as co-dissemination (p = 97), co-implementation (p = 41), throughout the research process (p = 39), earlier collaboration (p = 27), co-development (p = 22), future collaboration (p = 21), and initiation of the research project (p = 11). The most common activity was co-dissemination (p = 97), which involved knowledge dissemination organized in collaboration, or actors outside the university contributing with communication channels and activities. One of three proposals with meaning units classified into throughout the research process or future collaboration was granted, but only one of five with meaning units in co-implementation or initiation of the research project.

Reflections on collaboration

One of three proposals had meaning units classified into reflections on collaboration (p = 134), (see Table 3) including the codes access to data, competencies, and channels (p = 90), mutual learning (p = 57), and challenges and how to address them (p = 23). Meaning units about how collaboration would facilitate access to data, statistics, cases to study, competencies, organizations, networks, and communication channels for faster, broader, and more accurate dissemination and utilization were classified into access to data, competencies, and channels. These reflected primarily the benefit concerning to the researchers’ agendas and perspectives.

Meaning units classified into mutual learning included, for example, the importance of creating respectful relationships, exchanging information/ideas, and mutual value creation. Many of the benefits described were instrumental and practical while only a few described benefits such as shared insights/experiences and challenges such as tensions and barriers.

Meaning units classified into challenges and how to address them concerned how collaboration pitfalls could be identified and obstacles such as new problems or ethical dilemmas eliminated. Examples were how to engage with “hard-to-reach” groups, maintain commitments to long-term collaboration and manage actors’ diverse experiences, expectations, and agendas such as political and profit-driven interests:

Although differences in expectations, background and purpose for participating may provide obstacles and disagreement, the same factors and that very disagreement can also highlight important research themes that would otherwise have gone undetected and unaddressed… (Rejected program proposal, EquaLi).

One of eight proposals with meaning units in challenges and how to address them was granted compared to one of three with codes in access to data, competencies, and channels as well as mutual learning.

Other text fields

Many meaning units were not consistent with Forte’s instructions in Prisma for the text field collaboration, and thus classified into other text fields (p = 219) (see Table 3). When the applicants described one-way communication, with no other forms of collaboration, these meaning units were classified into knowledge dissemination (p = 91); one of four proposals had such meaning units. Despite Forte’s instructions that collaboration with researchers should be described in the text field, work plan, such meaning units occurred in one of four proposals, subsequently classified into collaboration within universities (p = 88).

Concepts describing collaborative research approaches in the proposals

The word search in the text field collaboration identified a multitude of different concepts. Most of them occurred once in one proposal, and none were common (see Table 4). Conceptual definitions and theoretical underpinnings for collaborative approaches were found in one of nine applications.

Table 4 Concepts for collaborative research approaches
Full size table

Assessments and decisions

The reviewers’ assessments under the heading Communication and Cooperation generated 414 meaning units that were consistent with Forte’s instructions in Prisma and 176 meaning units that were not consistent with the instructions (see Table 5).

Table 5 Meaning units from the review panels’ assessments (N = 381 proposals submitted to Forte during 2016), under the heading Communication and Cooperation in Prisma, classified into categories, subcategories, and codes [number of meaning units (u)]
Full size table

In the category evaluation, one-fifth of the meaning units were classified as positive evaluation (u = 89). Meaning units such as “OK” or “Communication and collaboration are described satisfactorily” were classified into sufficient collaboration. All assessments that pointed to insufficiencies in the proposals were classified as negative evaluation (u = 77), with meaning units such as “Not satisfactorily covered” or “Poorly specified and brief comments only” classified as insufficient collaboration. Cooperation with public authorities (u = 41) was the most common form and partner outside the academia. Only a few about collaboration with NGOs (u = 16) or laypeople (u = 6).

Several of the comments lacked content about collaboration or included content that did not comply with the instructions (see Table 1). There was no content about collaboration with actors outside the university in three of ten assessments, classified as lack of content on collaboration (u = 176). Many meaning units concerned collaboration within universities (u = 62), international collaboration (u = 42), and cooperation with public authorities (u = 41).

Discussion

The present study shows that in the context of a Swedish governmental funding agency targeting health, working life and welfare, proposal content related to collaboration is fragmented and conceptually and theoretically immature. Noteworthy, collaboration does not appear as a crucial criterion for funding, and reviewers have not internalized current policies focusing on collaboration between actors within and outside the universities and do not refine their assessments according to given criteria.

The descriptive findings of how applicants respond to research funders’ requests for collaboration and reviewers’ assessments of these responses indicate that there is a diversity of interpretations of the collaboration criterion. The diversity of approaches to collaboration in research and the lack of explicit conceptual definitions and theoretical underpinnings (Graham et al., 2006; Powell et al., 2018) is a plausible explanation of the findings of our study. This is not necessarily a reflection that researchers do not describe collaboration efforts in their proposals, but rather an observation indicating that in practice, collaboration is treated as an integrated effort applying pragmatic rather than theoretical approaches. For example, conceptual definitions of collaborative research approaches were absent in most of the proposals. The diverse terminology, mainly used without any definitions, in the few proposals where such concepts were included, indicates that there is little attention to the codification of collaboration and coherent terminology for its characteristics. This is in line with findings from other studies showing that language use is changing over time and that there is a lack of common terminology for collaboration activities (Bjursell et al., 2016; Graham et al., 2006; Powell et al., 2018). According to Harder et al. (2013), the diversity of approaches and lack of common terminology can place invisible inherent constraints on collaboration for researchers from diverse disciplines and research fields, and various actors outside the universities. As recommended by Nguyen et al. (2020) a focus on collaborative processes might be more meaningful than making efforts to define labeling concepts.

The applicants describe the intentions and benefits of collaboration, which is in line with Perkmann et al. (2013) who identified that non-commercial research collaboration is closely related to traditional research activities and motivated to get access to resources supporting their agenda. Few descriptions in our material included the perspectives of actors outside the university, but descriptions of benefits and effects for them could probably be found in other parts of the proposals such as societal relevance. Most of the collaboration activities involving actors outside the university were planned for later phases and not integrated into the early phases of the research process. It seems as if many applicants and reviewers do not embrace the ambitions of multi-actor collaboration throughout the research process, which is in contrast with the tenets of existing collaboration concepts (Bammer, 2019; Dungan et al., 2019; Hessels & van Lente, 2008; Pedrini et al., 2018). Moreover, the growth discourse to strengthen the competitiveness of the Swedish Government’s research policy bill (2016) was lacking in the descriptions of the benefits of collaboration. The economic growth argument may be more valid in other research fields (e.g., Olmos-Peñuela et al., 2015) than in Forte’s areas of responsibility, focusing on social welfare. If there are appreciable differences, there is an important message to research policy acknowledging and cross-fertilizing diverse concepts and approaches so that researchers in various research fields can incorporate them into their work.

When collaboration is a requirement in a call there is a risk that the importance of collaboration is distorted in the proposals, from being a means and a democratic rights-based approach to generating new and relevant knowledge that can be applied in practice to a goal in itself for obtaining research funding (Bjursell et al., 2016). If collaboration plans are pure rhetoric or impression management, the participatory efforts may not go beyond the realms of tokenism or the purpose and ethics may be thwarted by cultural failures (Dungan et al., 2019; Rolfe et al., 2018). Perkmann et al. (2013) argued that the assumption that more collaboration is better may increase the risk of failure and that a better strategy for research policy is to distinguish the conditions under which collaboration leads to research quality and societal benefits.

An unexpected result was that applications acknowledging collaboration issues such as discrepancies, tensions, and obstacles and how to address them were rejected to a greater extent. In our opinion, such proposals should rather be prioritized as they might generate much-warranted knowledge. In participatory efforts, it is necessary to balance positive and negative consequences, applying critical approaches (Bjursell et al., 2016; Stilgoe et al., 2014). Research focusing on benefits as well as challenges related to collaboration with actors outside the university is still scarce but certainly needed (Iwarsson et al., 2019).

Concepts such as PPI and RRI do not appear as prominent in research within Forte’s areas of responsibility and very few review panels commented on collaboration with the public. Thus, the democratic rights-based perspective with ideas of empowerment, shifts in power structures, and research with the public instead of research for the public were not prominent in the material. The involvement of the public to accomplish societal impact is an under-developed area (Powell et al., 2018; Pedrini et al., 2018). According to Bammer (2019), decision-makers who have the authority or capability to implement recommendations, emanating from research as well as those ultimately affected by the research, need to be included when research problems are complex. Everyone is aware that collaboration weighs heavily in the research policy debate, which is reflected in research councils' calls for research grants—not only in Sweden but also internationally (Macq et al., 2020). As to the importance of collaboration aspects of research proposals in the assessment and decision process put forward by funding agencies, our findings indicate that the influence on the funding decisions is minor.

Apparently, the applicants had diverse interpretations of how to separate collaboration content from other parts of their proposals. Many researchers are aware of the need for transdisciplinarity knowledge production to bridge the research-practice gaps. Collaboration within and outside the university is an obvious and integral part of their research activities. Against this background, it is a challenge to disentangle the integrated research approach as per the instructions in the online portal and at the same time describe it in a qualified manner in line with current developments. A reflection based on practical experience of writing proposals in Prisma is that as an applicant, you necessarily adapt to instructions and the maximum number of characters allowed in each part of the proposal. With several parts where the content can be overlapping, it is both a limitation and an opportunity that the text must be divided into different (smaller) parts of the proposal. The opportunity lies in that to maximize the use of space, for merely practical reasons the applicant can choose to place text in a different part than what would be the most appropriate, which may explain that some content appeared as non-consistent. Consequently, fragmentation based on instructions contradicts the idea of integrated research approaches. Another explanation of the diverse interpretations of what to include in the mandated parts of the proposal is that applicants might be unfamiliar with the ongoing shifts, trends, and developments of collaboration, the different definitions, and interpretations as well as possible benefits and challenges in different research fields.

Moreover, it is noteworthy that many of the assessments lacked content based on the collaboration criterion. However, assessments relating to this criterion could be found under review headings that were not analyzed in the present study, but our findings indicate that other assessment criteria dominate as the ground for recommendations for funding. Research quality was most likely the top criterion, but the findings of our study nevertheless indicate that the reviewers had not internalized the current policy priority on collaboration. In addition to improving the instructions and composition of parts of the proposal, the instructions and guidelines for reviewers must be more explicit and help them to identify and assess according to the predefined criteria. The findings indicate that many proposals had descriptions of more asymmetric power relationships than indicated by the term collaboration (i.e. teamwork with a shared objective). As an alternative, it may be better to use umbrella terms such as participation or involvement. A possible way forward may be to deepen a united codification of participation and adopt the Participation Framework proposed by Harder et al. (2013), serving as a conceptual aid covering the depth, breadth, scope, and effects of participation.

Methodological reflections

Governed by the commission from Forte underlying the present study, only selected parts of the proposals and assessments were analyzed. Forte’s application and review structure in Prisma contains other parts, some that are considerably more extensive and more central to whether a proposal should be rejected or granted. Although collaboration was our focus, there are limitations related to the concentration of one of the different but related parts that compose a coherent proposal. This obviously limited our possibilities to make analyses that hold more analytic value. Taking this further, after the completion of the present study we performed a deepened analysis of 17 complete and successful applications from the same material, which allowed us to draw more impactful conclusions and arguments (Hultqvist et al., 2021).

The content of the material did not fully correspond to the preliminary categories we initially chose for a deductive approach, indicating that these were not completely valid for the present material. This stated, the findings might give a somewhat superficial impression, but the material analyzed had such a character. Further in-depth analyzes of the extensive material now organized in an NVivo database could lead to important additional insights. Further studies are warranted to deepen the understanding of the findings of the present study and diverse collaborative research approaches and efforts.

The two authors’ teamwork throughout the analysis strengthens the validity of the findings. In addition, the dialogue in the feedback workshop, the continuous dialogue with Forte employees and the external review of our report in Swedish contributed to the validity and strengthen the study’s trustworthiness and credibility.

Our study was conducted in a country‐specific context, while globally, research and innovation policies are diverse and different (Coles et al., 2014). Still, a common denominator is to address humanity’s challenges by striving for sustainable and resilient development (United Nations, 2015), and we consider it likely that the findings of the present study will elicit interest internationally.

In conclusion, this study highlights that collaboration is described in a scattered manner in research applications. Elucidating how changes in research policies are adopted, operationalized, and assessed, the findings highlight that to fulfil political ambitions and raise the awareness of such matters among applicants and reviewers, funding agencies should make efforts to develop application and assessment guidelines and criteria based on definitions of key terms anchored in the international knowledge frontier. Further, to encourage applicants to produce coherent descriptions of participatory approaches and collaboration we recommend funding agencies allow applicants to elaborate on their plans in coherent rather than fragmented manners.