Keywords

Introduction: Forms of Impact, Ways of Mattering

Social studies of science have long taken an interest in the use of science in policymaking, including efforts among researchers to have the knowledge they produce taken up in settings where political decisions are made (Camic et al., 2011; Pielke, 2007; Weiss, 1980). Recognized as a pivotal way for universities to matter, this feature of science–policy interaction is increasingly conceptualized in terms of “knowledge brokering,” understood as a practice that unfolds at the science–policy interface and through which knowledge is made actionable (e.g., Bandola-Gill, 2023). Fertile soil for knowledge brokering to prosper is found in ad hoc commissions and the governmental reports they produce, used as they are to substantiate political reforms. This interface, then, provides a golden opportunity for universities to matter by mediating, linking, and connecting stakeholders of knowledge, but also by crafting or recrafting knowledge objects in ways that increase their prospects of impacting policy. This is the general theme of the present study.

In the reasonably specific culture of Swedish national policymaking (see Pierre, 2016), the information on which policy is based is readily available since the stock of knowledge in any given subject area is regularly presented in governmental reports, known as Statens offentliga utredningar (SOUs). It follows that the reference lists in SOUs are sites at which knowledge uptake is rendered salient. Among other things, SOU reference lists flag the extent to which national university-produced research contributes to knowledge creation relevant to national policymaking. This question, which is explored at length in this chapter, can also be framed as follows: Do Swedish universities produce policy-relevant knowledge and thereby have an impact on—or matter to—Swedish policy?

Conspicuously, mattering-through-brokering in this context can be tangible, as is the case when researchers are practically engaged in institutionalized processes of knowledge production and uptake. For example, as members of or experts in commissions, they may play roles as mediators and as intellectuals traversing science–policy boundaries (Osborne, 2004). However, this form of knowledge brokering is not the focus of the present chapter (but, see e.g., Thune et al., 2023; Salö, 2021a; Wisselgren, 2008). Instead, here we take an interest in forms of knowledge brokering that do not require direct, in-person involvement on the part of researchers. We take stock of the fact that brokering may also be found in events of uptake and occurrences of knowledge utilization where agents involved in commission work draw on knowledge objects—here texts—to craft policies and buttress political action. Here, as it were, the texts are doing the brokering without any practical engagement of their authors.

As we seek to highlight here, it is a fallacy to assume that the first, direct mode of engagement is rife with agency, whereas the second, indirect one is merely a question of passive knowledge uptake that the knowledge producer cannot influence in any way. This insight is crucial because it breaks with the image of the impact lottery wheel, which suggests that mattering lies entirely beyond the control of the knowledge producer (see also Perez Vico et al., 2024, this volume). On the contrary, as we hold, scholars who seek policy impact as an indicator of mattering can increase their chances by endorsing indirect, text-based knowledge brokering as a feature of their publishing practices. This entails, in short, producing impactful text types, which requires knowledge of the qualities of texts that are taken up in policy.

To make this point, this chapter presents a study of Swedish governmental reports (SOUs), focusing mainly on their reference lists and the items they contain. The objective is, firstly, to present an analysis of language use and reference type in the reference lists of Swedish SOU reports and, secondly, to discuss the dynamics of knowledge production and uptake as revealed through the lens of the concept of knowledge brokering. To this end, we present an analysis based on a sample of recent reports (2018–2021) from 10 government ministries. Empirically, we seek to interrogate the characteristics of the SOU reference list in terms of language use and type of sources cited. In so doing, we additionally seek to advance a discussion on how the study of SOUs may inform contemporary debates on the societal impact of research, particularly vis-à-vis questions of agency in knowledge brokering, production, and uptake.

Ultimately, our interest in issues of reference type and language serve to problematize the conception among researchers—real or perceived—that English-language journal articles are all that matter in scholarly production. On the contrary, as the chapter shows, governmental reports across the ministries, dealing with nationally embedded matters as they do, base their claims and subsequent policy recommendations on expert agency reports, public works, and academic works, the vast majority of which are so-called gray literature published in Swedish. In this regard, using national languages for written academic purposes is a brokering device—the same goes for producing reports and other forms of literature defined as gray. Some implications of this insight will be discussed in the present chapter, supported by old and new insights on agency in processes of value creation. In addition to the analysis of language and reference type, we provide an example of a more explorative in-depth approach in which machine reading and citation analysis are used to study the knowledge base of one particular governmental report. Such detailed analyses—although still limited in scope due to data availability—may open possibilities for further inquiries into the actual knowledge claims that support policy.

We begin by providing contextual background on regimes of knowledge politics, increasingly uniform patterns in scientific production, and the tradition of science–policy interaction in Swedish political culture. Then we present our methodological procedure and findings. Toward the end of the chapter, we discuss the implications of our results as they relate to the concepts of agency and knowledge brokering. We close by presenting concluding remarks.

The Politics, Uniformation, and Interaction of Knowledge

Successive Regimes of Knowledge Politics

As knowledge-political regimes gain and lose currency, the tasks of universities are reimagined, reappraised, and accordingly, reformulated. Notably, while regimes of knowledge politics may supersede each other, they standardly coexist side-by-side, leading to situations where friction is unavoidable (cf. Langfeldt et al., 2020). As a case in point, within the politics of science globally, the pendulum has long swung between competing ideas about universities’ purposefulness, usefulness, and entire raison d’être. On the one hand, there is the commercial side of innovation, through the logic of which science is envisioned chiefly as a lever for growth and a participant in international competition (e.g., Benner, 2018). On a par with the “excellence regime” hailed within this vision, impact is for the most part conceived as intra-scientific recognition, where productivity measures and citations by peers, for example, are used to gauge the impact of scholarly work and, by extension, the importance of such work (e.g., Aksnes et al., 2023; Langfeldt et al., 2020; for Swedish developments, see Nelhans, 2022, as well as Müller, 2024, this volume).

On the other hand, there is the broader and arguably irreconcilable vision of knowledge usability, where the quest for commercialization is deprivileged in favor of a social emphasis: one within which universities are called upon to matter in new ways and in relation to publics beyond their own realm. Indeed, in the meta-scientific debates surrounding the academic realm, there has been a call to reimagine the subsistence of university life, to reorient toward extra-scientific modes of knowledge exchange, and to make research accountable to end users rather than to scientific peers only (e.g., Burawoy, 2005; Sarewitz, 2016). Reappraising the bottom-line value of universities’ knowledge production also entails reappraising how research is evaluated. Initiatives such as the Coalition for Advancing Research Assessment (CoARA), the Declaration on Research Assessment (DORA), the Metric Tide, and the Leiden Manifesto have emerged as part of a more significant movement for broadening how research is evaluated and valued (Rushforth & Hammarfelt, 2023). Alternative metrics (including social media metrics) as well as the launch of “impact stories” in the UK are additional signs of how a broader discussion on the assessment of academic research is gaining ground (Bornmann, 2014; van Noorden, 2015). In these accounts, the broader influence of academic research is emphasized at the expense of narrow performance measures, such as citations or impact factors.

Features of such global dynamics have also seeped into Swedish science policy. By the 1990s, science policies had become increasingly entangled with growth policies; the direction thus chosen was founded on global and regional competition and subsequent attempts to maximize the worth of the universities’ research activities. Among other things, under the pressure of “evaluative neoliberalization” (Benner, 2023), these processes ushered in a culture of quality managing, auditing, and measuring designed to bolster excellence as gauged through the increasing use of metrics.

Since the 2010s, however, there have been reasons to suggest that much of this was fundamentally upset by a new knowledge-political regime (Sörlin, 2015, 2021). Among the buzzwords signaling the government’s prioritizations in the area, “excellence” was saliently replaced by “collaboration” (samverkan) between 2012 and 2016 (Hammarfelt, 2021b; see also Benneworth et al., 2015). Scholarship has followed suit, as made evident by work seeking to rethink old conceptions of what impact might mean, particularly in the human and cultural sciences (e.g., Benneworth, 2015; de Jong et al., 2022; Muhonen et al., 2020). Several Swedish studies have been able to show empirically how ideas stemming from humanities research practices have gained broader circulation in society to eventually become more or less direct sources of policy and change these societies in fundamental ways (Bertilsson, 2021; Salö, 2021a; Salö & Karlander, 2022; see also Perez Vico et al., 2024, this volume).

The reliance on evidence-based research to ground decision-making is perhaps more commonly related to other professional areas, particularly technological, medical, and environmental sciences (Youtie et al., 2017). In evaluating the societal impact of research, government agencies, and funders have sought to identify and measure references in policy documents and patents as indicators (Bornmann et al., 2015; Hammarfelt, 2021a; Lewison & Sullivan, 2008; Wilsdon et al., 2015). These have been proposed as professional metrics to distinguish this form of societal impact from that measured in social media, often under the heading of altmetrics. In contrast to the altmetrics, which often fail to differentiate between various types of traceable activities, whether they occur on the internet, within policy realms, through patents, or in clinical settings, the concept of professional impact centers on references within documents published by reputable professional or governmental organizations. This attribute lends these references a degree of permanence and stability, enhancing their reproducibility (Nelhans, 2016). Insights yielded from this research are that referencing in national guidelines is relatively local, with a high share of references to research published in the same country, but, at the same time, that there is a distribution of references to research published by smaller units and centers outside of the metropolitan regions where traditional citation impact is more considerable (Nelhans, 2016).

Two Kinds of Scientific Uniformity: Text Types and Languages of Publishing

The extent to which the most recent developments within the politics of science have influenced the practices of scientific communities is unclear. Many within those communities still seem inclined to engage in knowledge exchange chiefly with their peers. This is evident through the types of texts they produce. Even by the early 2000s, Kyvik (2003) had shown how the publishing behavior of Norwegian scholars changed between 1980 and 2000, displaying an increased focus on publishing directed toward international audiences and a greater preference for scientific articles in international journals to achieve such ends. Although text types such as monographs and edited volumes still predominate (Engels et al., 2018), the journal article is doubtlessly the predominant text type for scientific exchange in the contemporary age (e.g., Kaltenbrunner et al., 2022; Savage & Olejniczak, 2022).

The standardization of publication types is, in turn, tangled up in an older trend of linguistic uniformity, which currently presents itself as English language dominance in scientific production (e.g., O’Neil, 2018). Across the language sciences, a general critique against “the spread of English” gained currency from the 1970s onwards (e.g., Fishman et al., 1977; Pennycook, 1994; Phillipson, 1992) and eventually came to center on “global English” in science and higher education, more specifically (e.g., Ammon, 2001). More recently, these have become debates to which scholars from many fields contribute (e.g., Gordin, 2015; Sivertsen, 2018). The prevalence of English in scientific production is by now well established.

Combined with the enhanced international orientation, then, it is more accurately English-language journal articles that increasingly prevail internationally. While there are major disciplinary and regional differences, the English-language journal article stands out as the current era’s supreme way of packing and subsequently communicating scientific knowledge (Kuteeva & Airey, 2014; Lillis & Curry, 2010). In the human sciences, too, there is a strong preference among scholars in the Nordic region to publish journal articles in English (Kulczycki et al., 2020). Notably, it may well be that this “preference” ought to be viewed as a form of compliance with schemes of knowledge valorization. Across national contexts, studies have pointed to the ways in which the use of metrics-based research evaluation systems impacts on scholarly agendas, inducing scholars to adopt international dispositions toward scientific communication and prodding them to orient increasingly toward producing journal articles published in English (Feenstra & Delgado López-Cózar, 2022). As an effect, as Swedish studies have shown, some publishing behaviors have been encouraged—particularly the production of peer-reviewed English-language articles in indexed journals. In contrast, little profit has been ascribed to other text types and languages (Hammarfelt & de Rijcke, 2015). Critical concerns about such matters have been raised first and foremost by humanities scholars, whose traditional academic behavior has begun to be perceived as no longer desirable (Salö, 2017).

Again, however, regimes of knowledge politics are subject to change. Recently, driven by a stronger focus on societal impact, The Helsinki Initiative (2019) presented a plea for scientific multilingualism that includes scholarly communication in national languages. While the debate is almost as old as science itself (Gordin, 2015; Salö, 2017), the fact that this call is re-energized now is probably not a coincidence. As described earlier, there is currently a push for making universities matter, with ripple effects in science policies as well as across the sciences, stressing how mattering has deep societal ramifications. As highlighted by The Helsinki Initiative (2019), issues related to language choice in scientific production play a prominent role in such dynamics because of the value held by local languages in processes of impact creation. Even though the role of language in processes of science–policy knowledge utilization remains under-researched (but see Droz et al., 2023; Ringe, 2022), debaters have long argued that, despite its benefits in transnational communication, the use of English can in fact pose obstacles when it comes to national communication, particularly vis-à-vis processes of national policymaking. In the Nordic region and elsewhere, observable patterns in language choice reveal a gap between scholarly knowledge production and, among other things, policy knowledge demands (e.g., Hultgren et al., 2014; Salö, 2018). Accordingly, actors such as the Nordic Council of Ministers (2007) have advocated so-called parallel language policies as a remedy (see Holmen, 2017; Gregersen, 2018). Such policies seem welcome. For instance, mapping the users of national open-access journal articles in Finland, Pölönen et al. (2021) showed that students, citizens, and politicians favored Finnish-language publications compared to researchers, who preferred foreign-language articles. This would indicate that scientific articles published in national languages have a broad audience extending to the policy sphere.

Nordic Modes of Science–policy Interaction

The Nordic region is interesting not only because of its firm embrace of English in science but also due to its widespread use of scientific expertise in policymaking (Lundqvist & Petersen, 2010). In fact, the region is known for its well-established, even institutionalized, traditions of knowledge brokering. Sweden, the context for this chapter, exemplifies this well insofar as it has long been internationally known for its rational procedures of grounding political reforms in research-based knowledge (e.g., Anton, 1969; Castles, 1976; Eyerman, 1985). While pivotal conditions have changed over time, this reputation lingers to this day. Tellingly, The Oxford Handbook of Swedish Politics (Pierre, 2016) devotes several chapters to more or less Sweden-specific modes of policymaking, focusing on traits that remain as well as those that have been forsaken (e.g., Mattsson, 2016; Pettersson, 2016). Among many foci, a central component of such discussions has been the Swedish system of governmental commissions as a distinctive art of knowledge-infused state governance (Johansson, 1992; Premfors, 1983). In short, Swedish reform politics have long relied on an institutionalized procedure of reaching consensus by appointing expert-led commissions prior to making major political decisions. Commentators have differed in their characterization of this system and the agents involved in it, ranging from depictions of expert trust (Lundqvist & Petersen, 2010) to more critical perspectives on welfare state strategies of realizing aims through the utilization of reform technocrats (Lundin & Stenlås, 2015). Irrespective of divergent views, it is a well-established viewpoint that governmental commissions have played, and continue to play, a vital role in policymaking and processes of knowledge uptake in Swedish politics (e.g., Trägårdh, 2007).

As viewed here, the commission system, and the writing of SOUs within it, is an example of a “science–policy interface” (Bandola-Gill, 2023). The engagement of experts, policy intellectuals, or reform technocrats has provided university-based scholars with the opportunity to transgress the science–policy boundary (Wisselgren, 2008). Indeed, governmental commissions have often recruited expertise from the universities, not least from the humanities (Dahlberg, 2021). It follows that ad hoc commissions make up paramount knowledge arenas that create what de Jong et al. (2022) call “enabling conditions” for researchers to matter. Yet, if researchers are interested in having their knowledge taken up in processes of commission work, they have to present it in ways that yield impact—hence the questions posed in this study. These dynamics, as we will outline presently, pertain to questions of agency.

Key Conceptualizations: Agency in Knowledge Production, Uptake, and Brokering

Human agency, briefly put, has to do with the capacity to act and, more specifically, the acting agent’s power to exert control over actions and their effects. It often concerns the acting subject’s ability to act deliberately, freely, or purposefully and is thus pitted against the view of action as constrained by the force of structure. In science studies by the mid-1970s, Gibbons and Johnston (1974) drew attention to matters of agency in knowledge exchange processes. Focusing on technological innovation, they argued that scientists do not passively add knowledge to a pool of knowledge but can

play a vital role in perceiving the application of particular knowledge to specific problems, and translating and transforming the results of scientific research into a form in which they are directly usable in the industrial environment. (Gibbons & Johnston, 1974, p. 242)

Through the seminal work of Gibbons and colleagues, the same emphasis later became a feature of so-called mode 2 knowledge production, characterized among other things by how knowledge develops in the context of application (Gibbons et al., 1994). While later critically debated and revisited, the same basic point also remains valid regarding the role of science in social innovation (e.g., Camic et al., 2011). Increasingly, scholarship stresses how researchers create their own “acting space” (Perez Vico et al., 2024, this volume) and are thus involved in influencing the market conditions that value their own knowledge products (Salö, 2021b).

On the flip side, uptake, too, is an agentive activity. As Rip (1998) noted, paraphrasing Gibbons and Johnston (1974):

Eventual take-up of knowledge (“application”) is an activity by itself, not the effect of a knowledge push. And even more important, it is almost always indirect: knowledge products are delivered into a knowledge reservoir, carried by what one might call an epistemic community, and knowledge users pick up their own new combinations from the reservoir. (Rip, 1998, p. 14, italics in the original)

Decades ago, Weiss (1980) was able to demonstrate how “creeping” social knowledge slowly informs policymakers, who more or less selectively and deliberately make use of it in decision-making processes. Knowledge, accordingly, is not passively taken up but instead actively utilized and fed into decision-making processes. Recently, such perspectives have gained salience and currency. Notably, a focus on “productive interactions” (Spaapen & van Drooge, 2011) brought forth novel views on social impact assessment and spurred further thinking on the dynamics of impact pathways (Muhonen et al., 2020). Often, questions of agency have occupied a central place. For example, drawing on interviews with civil servants, Tellmann and Gulbrandsen (2022) depict the users of research as strategic actors entrenched in the productive science–policy interactions that yield societal impact. The fact that users of knowledge are not passive recipients of knowledge, following de Jong et al. (2022), is in itself an enabling condition for productive interactions—and thus impact—to occur.

In sum, then, attention to agency has often served to problematize the view of the recipient of knowledge as passive and unable to exert influence over knowledge production. By the same token, proponents of agency are prone to critique the view of producers of knowledge as active but nonetheless unable to steer conditions of desirable knowledge reception. On the one hand, emphasizing the agency of researchers, it is clear that scholars can direct their efforts toward a specific context of application or act to render their knowledge policy-relevant, visible, and actionable. On the other hand, policymakers and other knowledge users are positioned to strategically select what knowledge to draw and act upon (Pielke, 2007). Arguably, these dynamics largely pertain to tangible forms of interactive engagement. Discussing productive interaction, Tellmann and Gulbrandsen (2022) make a fruitful distinction between direct, personal interactions between researchers and stakeholders and indirect interactions through text between the same set of agents. Both of these modes of interaction require knowledge brokering, broadly understood as a process of transferring research-based knowledge into action (Ward et al., 2009). The process as such incorporates several steps, including the identification and redistribution as well as the rescaling and transformation of knowledge (Meyer, 2010) whereby knowledge brokers “link the producers and users of knowledge to strengthen the generation, dissemination and eventual use of that knowledge” (Bielak et al., 2008, p. 203). Following Meyer (2010), brokering renders knowledge more robust, usable, and accountable throughout such translational activities.

As understood here, following Bandola-Gill (2023), knowledge brokering comprises a wide set of mediating practices enacted by researchers in order to make academic knowledge useful in policy settings, thus encompassing not only the firsthand involvement of animate brokers but also—obliquely—the intermediary text artifacts they produce. This conceptualization is ultimately important because it invites us to explore whether some types of published works—brokering text types—are more impactful on policy than others. Swedish SOUs, the textual outcome of governmental commissions, provide a context concerning such questions.

Methodological Procedure

In Sweden, governmental commissions are appointed and subsequently published by the responsible ministry, each of which is headed by a minister and responsible for several government agencies. Each government is free to create, remove, or merge ministries, meaning their composition varies over time. In the period when the present study was conducted, the government office housed 12 ministries. Apart from the Prime Minister’s Office, these were the Ministries of Culture, Defense, Employment, Education and Research, Enterprise and Innovation, Environment, Finance, Foreign Affairs, Health and Social Affairs, Housing, Infrastructure, and Justice. Whenever issues arise within their respective areas of responsibility, commissions are thus appointed.

We selected two recent governmental reports from each of the 10 ministries. This choice was made as we suspected that different policy areas might employ scholarly sources to various degrees and in different ways. Regarding the selection criteria, we chose reports published between 2017 and 2021 with reference lists. In cases where we had several SOUs to choose from, we selected those that contained a reference list rather than those using footnotes or endnotes. Hence, our sample for the analysis was not chosen randomly. The ministries of Foreign Affairs and Enterprise and Innovation did not have two SOUs that matched these criteria and were therefore excluded. The resulting material thus consisted of 20 SOUs containing 2787 references. Hence, the chosen dataset was purposely selected for the explorative approach of this study, and it cannot be automatically assumed that it is representative of SOUs in general. Given the small sample, two per ministry, and the significant variation in how the different SOUs are structured, it is difficult to draw firm conclusions regarding how representative they are for the knowledge uptake of each ministry. A description of the complete dataset, including the title of each report, can be found in Appendix 1.

Analyzing sources used in SOUs poses challenges that do not occur with more fixed text types, such as journal articles, due to variations in style and completeness. References might be given in various places, and there might be multiple, partly incomplete lists for each SOU document. Such inconsistencies make the automatic detection of references difficult. We therefore opted for a semi-automated approach in which parts of the material were identified using software capable of matching Digital Object Identifiers (DOIs) to the reference lists in the SOUs. However, this method only allowed for identifying scholarly works that had a DOI, mainly journal articles and scholarly reports and, to a lesser degree, monographs and chapters in edited books. We therefore supplemented this method by manually detecting and categorizing references using a predefined set of characteristics. This methodology required us to limit our analysis to a few key features—language and scholarliness—of the extrapolated references that jointly determined their text type.

By “text type” we refer to what is elsewhere termed “academic genres” (Berkenkotter et al., 2012) or “literatures of science” (Hicks, 2004). Thus, we use this notion to differentiate between text families based on their envisioned uses and publics as well as the conventions they consequently adhere to. A vital factor here is the degree of “scholarliness” they display, where scholarliness is taken to be an effect of their procedures of production, intended readership, aspirations of scientificity, et cetera. As opposed to their non-scientific counterparts, scientific text types are characterized by their adherence to methodological and theoretical standards of the scientific communities. Their degree of scholarliness is thus estimated to be high.

For the present study, based on such assumptions of text types and their degree of scholarliness, we classified the references into two reference types: scholarly and non-scholarly. Assessing the degree to which references are to be regarded as scholarly was not without difficulties. Obviously, articles in scientific journals and books by academic publishers are considered scholarly. Dissertations belong to this category as well. What is evident in this type of document, however, is the importance of what is often defined as “gray literature,” which refers to various documents, often published or unpublished reports, that are not considered scholarly but may nonetheless contain content that has been academically produced (Börjesson, 2015). “Gray literature,” consequently, is notoriously difficult to define. Reports of various kinds are a common text type within this category. Here, we defined reports published by academic institutions (e.g., universities) as scholarly, while reports commissioned by government agencies or NGOs were defined as non-scholarly. Hence, the definition used here matters if the sender or producer can be identified as a legitimate source of academic knowledge. Overall, this implies that our estimate regarding scholarly influence is probably lower than the actual “academic” impact (if this is possible to estimate).

Items included in the reference lists that were identified as scholarly literature, usually English-language journal articles and possibly including a DOI, were submitted to the Simple Text Query Form [https://apps.crossref.org/SimpleTextQuery]. This service by CrossRef uses heuristics to match plain text references to the publication’s persistent DOI and attaches an identification code to each matched publication. The results were then further analyzed using descriptive statistics and were checked using the bibliographic database Web of Science. The DOI matching was not exact, and SimpleText could not find a match for every reference. This was particularly seen when instead of a publication year it said “forthcoming” or when comments in Swedish were intermingled with the reference text. Therefore, the reference lists used for matching were manually scanned, and all entries that looked like a reference that could include a DOI were singled out. Examples are a reference to an article in a scholarly journal, a cited document with a title in English with vague or no publication information, and a preprint that was cited and subsequently published. The DOIs for these entries were subsequently searched for manually using Google. This resulted in a reasonably low number (~10–15) of “extra matches” included in the analysis.

Findings

In what follows, we present the study’s findings. This is done in two steps: first, we outline the broader patterns of language choice and degree of scholarliness in the SOU reference list, then we take a closer look at their occurrences and networks of scholarly references in one particular SOU.

Variations in Language Use and Reference Type

We began by classifying the items of the SOU reference list in terms of language and, adopting a binary approach: whether they were scholarly or not. Regarding the language of sources, we found a clear dominance of Swedish-language titles (Fig. 1).

Fig. 1
A pie chart of the percentage distribution of languages used in the cited sources. Swedish, 7513. English, 23.82. Danish, 0.47. Norwegian, 0.43. German, 0.07. French, 0.04. Italian, 0.04.

Language use in cited sources

Full size image

Notably, there is a great variation across governmental reports regarding language (see Appendix 1). In Sverige och bankunionen SOU 2019:52 [Finance], 77% (117) of the references were in English, and in Vägen till en klimatpositiv framtid [Environment] SOU 2020:4, 48% (125) were in English, while all 95 references in Ersättning till brottsoffer, [Justice] SOU 2021:64 were in Swedish. Law is indeed an interesting field as the definition of what a “scholarly publication” is (e.g., a law commentary) is an item of discussion (cf. Kaltenbrunner & de Rijcke, 2017). Moreover, it is a field that, for obvious reasons, is dominated by publications in the national language (Salö & Josephson, 2014, pp. 281–282).

Next, we analyzed the types of sources cited, focusing on whether they were scholarly or not. Our analysis shows that a majority of sources were deemed non-scholarly (78%). Such infrequent citing of scholarly publications may be interpreted as a rather low uptake of academically produced knowledge in policy. However, it should be noted that the number of scholarly publications cited in SOUs (22%) might not accurately estimate the actual influence of academic sources. One reason is the frequent citing of other SOUs and various reports from government agencies and NGOs. While these sources are not scholarly in themselves, they often build on academically produced knowledge. Hence, we suspect that reports and summaries may be a text type that is more easily translated into a policy setting. Overall, we postulate that academic knowledge might be nested, or brokered, in such a way that its actual influence is not fully reflected in the citing of scholarly publications. As brokering involves more than a simple transfer of knowledge—Meyer (2010) uses the concepts of “travel” and “transformation”—the original sources for a claim might not be readily recognizable when traveling across contexts. Hence, reports and summaries of various kinds which are frequently cited in SOUs may in turn build on scholarly sources, and academics rather often write them. Therefore, the whole chain of knowledge translations and brokering that precedes the use of academic knowledge in SOUs needs to be investigated further. It might be suggested, for example, that the closer we come to the actual policy decisions, the more “hidden” the specific sources for the claim will be. Yet, such a hypothesis would require a more thorough analysis than the one provided here.

However, the overall result hides large variances in terms of both language use and use of scholarly sources. For example, the governmental report För flerspråkighet, kunskapsutveckling och inkludering—modersmålsundervisning och studiehandledning på modersmål, SOU 2019:18 [Education] stands out with 60% (146) of references being classified as scholarly. However others, like Immunitet för utställningsföremål, SOU 2021:28 [Culture] (0 out of 55 references) and Sveriges säkerhet—behov av starkare skydd för nätverks- och informationssystem, SOU 2021:63 [Defense] (1 out of 77 references), barely make any references to scholarly sources at all. At the SOU level, the share of scholarly references varied between 0 and 60%, with an extensive spread between documents (Standard deviation = 0.15, Median value = 12.2%). This variation makes it difficult to draw generalizable conclusions regarding the general use of scholarly sources in SOUs from this selection. Instead, it seems to suggest that how academic knowledge is used—if it is used at all—differs considerably depending on both policy area and the specific topic of each SOU. Moreover, it should be highlighted that a substantially larger study, involving considerably more SOUs for each department, would be needed to draw any generalizable conclusion regarding the degree to which specific policy areas rely on scholarly publications when writing SOUs.

Detailed Analysis of Scholarly References

Above, we have used the bibliographical data provided by the SOUs to discern general patterns in their use of sources. However, focusing on Digital Object Identifiers makes it possible to perform a more detailed analysis of cited materials. The DOI gives each document a unique identification number, making it traceable across platforms and databases. Such markings allow for tracking publications across social media (altmetrics), or as in this case, constructing a network analysis of citations. This type of analysis can help us grasp the characteristics of the sources displaying a high degree of scholarliness.

As indicated above, 612 of the total 2,787 references identified were considered scholarly. Of these, 338 references were matched with a DOI. Aggregated at the ministry level, some general aspects can be outlined. The SOUs were derived from 10 different ministries, with two SOUs being analyzed for each. The highest share of scholarly references was found in Education and Research (49%) followed by Finance (24%), with Culture (3%) and Defense (7%) recording very few references to scholarly work (Table 1). As noted above, these percentages are based on a small sample, and larger studies would be needed to substantiate these findings.

Table 1 Share of scholarly references at the ministry level, two SOUs analyzed for each department (see Appendix 1 for a full list of SOUs, including titles)
Full size table

In the next step, we tested whether references, matched through DOIs, could be used to visualize the academic knowledge base of a particular SOU. For this purpose, we selected one of the two SOUs from the Department of Education and Research: För flerspråkighet, kunskapsutveckling och inkludering—modersmålsundervisning och studiehandledning på modersmål [in English, For multilingualism, knowledge development and inclusion—mother tongue instruction and mother tongue study guidance], SOU 2019:18. This SOU was chosen because it had a relatively large share of documents with a DOI. Moreover, one of the authors (LS) could provide us with insights that would make it possible to interpret the findings. Without such contextual knowledge, bibliographical mapping tends to be reduced to a bunch of names across a colored background.

Using the Dimensions citation database (https://www.dimensions.ai/), we produced a citation network out of 45 references found in the SOU that had a DOI and were indexed in the database. These were then mapped using the VOSviewer software (Van Eck & Waltman, 2014) based on direct citation between documents (e.g., a link between the two is registered if document A cites document B) (Fig. 2).

Fig. 2
A color-coded network diagram of 45 identified references in the dimensions database based on direct citation between documents. The words are in a foreign language with words like palm 2018 and Francis 2009.

A network of 45 identified references in the Dimensions database (Digital Science, 2018) using VOSviewer (Van Eck & Waltman, 2014). Clustering based on direct citation (e.g., references from one document to another document in the same set. Based on references with DOIs from SOU 2019:18)

Full size image

The arrangement of this visualization relies on a citation network based on 45 identified articles, resulting in a citation map. In this map, each article serves as a “magnet,” attracting other articles based on the number of citations linking them. The size of each node corresponds to the number of citation links, and a clustering algorithm has been utilized to differentiate related documents based on their citation patterns distinguishing thematic similarities. In this cluster of references, we find several national and international authorities within the areas of multilingualism and education, language learning, and language policy. While many names are active in contexts other than Sweden, their research can be seen as theoretically relevant and sufficiently general to relate to the Swedish case. Several nodes in the cluster are made up of studies produced by Sweden-based scholars interested in mother tongue instruction; that is, the object at the center of the SOU in question. These studies, like those produced elsewhere, are relatively recent. This trait is in itself interesting, as mother tongue instruction has occupied a place in the national curriculum over the course of almost 50 years despite initially having a scant stock of knowledge to stand on (Salö, 2021b). As we will discuss critically below, this may serve as a reminder that the role of knowledge in state policy is far from a clear-cut, innocent affair (Benner, 2021). On the contrary, knowledge brokering involves power with effects across every stage, and on multiple scales, of science–policy interaction processes. Therefore, policy often moves faster than science (Salö, 2021b, p. 169).

Implications: Mattering Through Agentive Brokering

The reference lists of governmental reports (here, SOUs) are of interest because the work referenced in them forms part of the knowledge base used to support the various reforms and propositions presented by the government. This fact notwithstanding, questions relating to such forms of knowledge uptake seldom linger in debates about impact. To the extent they do, in scholarship and elsewhere, Swedish researchers sometimes complain that their possibilities for impacting policy have changed for the worse (e.g., Pettersson, 2013). Often, the reasons for this are placed on some perceived knowledge-receiving end and are explained by an increasingly marginalized commission system or other dynamics that seem to render science less important to draw on. While there is a kernel of truth in such accounts, it is striking that researchers only occasionally acknowledge the interlinkage between their modes of production and conditions for knowledge reception. Overtly resigned accounts also downplay their own agency in the process of science–policy interaction. Yet, as a wide range of studies have shown over the last half-decade—from Gibbons and Johnston (1974) to Tellmann and Gullbrandsen (2022)—agency deeply permeates the work of agents engaged in knowledge production and uptake. Because brokering is an interactive process—a practice locked in between production and uptake—the centrality of the agency is undeniable. Be it the result of direct or indirect interaction, this essentially means that policy impact and the knowledge on which it is founded is the outcome of a collaborative process that relies on the agency of all parties involved.

How might these insights gel with the account we have presented here? Admittedly, our findings do not speak straightforwardly to the question of agency as the analysis can only render broad patterns visible. In contrast, the study of real-life action requires other forms of inquiry. However, at some general level, we nonetheless hold that agency is a relevant lens through which to view matters of text-based impact. In this light, we hope that our findings will contribute to rejigging dominant understandings of impact and help signpost productive modes of interaction. While the case of personal interactions may involve brokers as a particular role in science–policy interplay (Osborne, 2004; Pielke, 2007), brokering in indirect and text-based interactions necessarily revolves around particular text types that mediate knowledge exchange between stakeholders in science–policy interactions and thus determine impact. In this latter sense, the patterns revealed here ought to be of interest, and possibly practical interest, among scholars concerned with impacting policy.

As we have sought to show, inquiries into governmental reports provide a way to empirically map processes of knowledge uptake and subsequent impact. The question at the heart of the chapter has revolved around the extent to which governmental reports rely on academic knowledge as observable through the sources in their reference lists. We asked, firstly, about the characteristics of SOU reference lists in terms of language use and type of references cited and, secondly, how such insights may inform contemporary debates on impact and the questions of agency they bring to the fore.

Overall, we have found that scholarly work on average makes up one-fifth (22%) of the total number of references cited. A salient pattern in our findings is the strong presence of gray literature, the vast majority of which is in Swedish, in the reference lists. Within this somewhat vague category, reports make up a large share. Reports typically compile existing knowledge within a given subject area; the task of writing them, accordingly, involves processes of selecting and interpreting research, and then repackaging such knowledge into a new text type, often produced in another language and aimed at another kind of audience. This process—which Meyer (2010, p. 123) describes as knowledge being “de- and reassembled”—can be understood as one of knowledge brokering, whereby the text type itself constitutes an affordance for political action. Gray literature is often brokering literature. Yet, as an actionable affordance, text types do not function alone, as language plays a part in the equation. In a country such as Sweden, where legislation and bureaucracy operate in Swedish, reports in Swedish can be said to constitute a brokering text type vital for knowledge-political aims.

Suppose there are clear patterns in terms of what text types and languages of publication are taken up in governmental reports. In that case, scholars who seek policy impact can adapt their publishing practices to fit such patterns. As an added value, there are possibly applied virtues in contemplating the implications of the patterns revealed. Exploring governmental reports might answer the question: What types of texts, defined at the interface between language and text type, are most likely to appear in the reports used to underpin the political decisions of the government and other state agents? Although we have used Sweden as a case, much of what we have dealt with here pertains to issues and developments of international reach.

While English-language journal articles doubtlessly facilitate scholarly exchange, other text types are significantly more impactful in science–policy interaction processes. The account we have presented here could be interpreted as one that says: If you want to impact policy, write reports in Swedish. This, however, would be a simplified imperative. Science–policy interaction, and the knowledge brokering it requires, is far more complex. For example, reports need knowledge with a high degree of scholarliness to draw on, and if the latter is not produced—often in English—there is little for reports to compile. Moreover, the impactfulness of reports might not reside so much in the text type as such as it does in the organization behind it, or, at times, in the scientific credibility of its author.

Moreover, junior scholars are likely acutely aware that reports are of less worth on their list of publications compared to other types of texts, and “brokering” initiatives may generally receive little recognition as an academic merit (Turnhout et al., 2013). However, what we are saying, harshly perhaps, is that it is a fallacy to assume that English-language journal articles will easily find their way into processes of policymaking and have an impact there. While our analysis shows that this may indeed happen, it is significantly more common to find knowledge-brokering text types in the reference lists of governmental reports. Given that Swedish-language reports make up a large share of the knowledge supporting political decisions, scholars seeking to matter politically might want to rethink their own publication agendas. By the same token, to the extent that the production of policy-relevant work is deemed desirable, institutional systems of evaluation may want to change evaluation schemes in which scholarly work is hailed as important, particularly in the social sciences and humanities (Sivertsen, 2019).

Based on the low share (22%) of scholarly text types in the SOU reference lists, one might conclude that the impact of academically produced knowledge on policy is marginal and rather restricted. Interestingly, however, our sample shows that the degree to which scholarly publications are cited varies considerably. Although other circumstances may have an influence, it appears that some areas of government tend to make extensive use of scholarly literature. In contrast, scholarliness is a less salient characteristic in others. Moreover, we suggest that the ample citing of gray literature (reports, reviews, etc.) may hide the influence of academically produced knowledge. If so, we might view the SOU as the endpoint of a process of translation through which knowledge is brokered to eventually enter the realm of policy. It might also be the case that the closer knowledge claims come to actual political decisions, the more invisible the origins of these claims become. This is one of many reasons why knowledge brokering ought not to be embraced uncritically; clearly, there are many potential problems and challenges linked to taking up intermediary positions (e.g., Kislov et al., 2017).

A conclusion would then be that SOUs are a good start for studying science–policy interactions. Still, to actually produce insights into how these interactions happen, we need to proceed beyond the SOU and tap into processes and relations that precede the production of these documents. We acknowledge that the ministries likely house distinctive policy cultures, where some are time-honored and nationally anchored (e.g., law), whereas others (e.g., climate and multilingualism) are newer and more closely interwoven with global networks of knowledge circulation. Likewise, we grant that the use of knowledge probably depends on the individual who assembles it and the mechanisms through which the ones producing these reports are chosen. While some things are known about procedures and practices of knowledge utilization in SOUs (Bringselius, 2021; cf. Thune et al,. 2023), the need for more research is pressing. In short, in anticipation of future research, there are reasons to maintain a critical gaze on how state agents control policy processes in ways that make their ostensible knowledge base a mirage (Bourdieu, 2005, pp. 104ff.; see also Benner, 2021, for a comment on the Swedish case more specifically).

The understanding of such dynamics requires the use of multi-methodological research. Within such an agenda, where ethnographic fieldwork is warranted, a range of other concealed modus operandi with clear implications for the present study may also be explored. For example, what characterizes the processes through which letters of instruction, so-called commission directives, are crafted and agreed upon, and to what extent do they have a steering impact on the policies that are eventually presented in SOUs? How are SOU commissioners selected? How do they roam or probe into the stock of knowledge concerning the topic at hand? To what extent do their cross-field movements involve the university realm, and to what extent do competing stakeholders such as think tanks play a role in policymaking processes?

One reason why questions of knowledge uptake and use seldom linger in debates about impact has to do with the lack of established methods for systematically analyzing policy impact. While we have sought to contribute to this end, we have at the same time strived to highlight that knowledge uptake and impact are complex processes, the study of which is rife with analytical challenges. Our study points to many of the difficulties of analyzing SOUs, with the unstandardized use of references being a major hindrance for larger systematic studies. Yet, our detailed example of mother tongue instruction shows that it might be possible, at least in selected cases, to find traceable links that can be used to study the uptake of scholarly publications in policy text quantitively. At present, such analysis is dependent on the existence of machine-readable DOIs, but the rapid development of software for detecting references may open possibilities for more detailed analyses of SOUs in the future. For example, one might analyze the age of the sources cited to grasp whether or not policies have been based on recent knowledge. It might also be interesting to study which researchers and institutions are cited by SOUs, and whether patterns in fields, institutions, or countries can be detected. Moreover, since all SOUs from 1922 to 1999 are available in digital format, there might be opportunities to study the influence of research (and researchers) over extended periods of time. At any rate, there is potential in studying governmental reports. Our ambition is to see the approach outlined, including its theoretical assumptions, applicable on a larger scale and to prod colleagues to develop it further.

Final Remarks

This chapter has explored indirect, text-based knowledge brokering through a study of Swedish governmental reports, analyzing their reference lists in terms of language use and reference type. We found that 78% of references were to non-scholarly literature, and that around 75% were Swedish-language references. This finding is interesting given that, for contemporary universities, the English-language journal article stands out as the predominant text outlet. While such uniformity has its advantages, the situation it yields requires knowledge brokers and brokering to navigate what we might call the language–knowledge–text type nexus. Insofar as mattering entails meaning something to someone, furnishing the sources of policy stands out as a fundamental way for universities and their scholars to serve such ends. The uptake of their research is an indicator of their knowledge products buttressing political decisions—good or bad—that potentially affect the lives of many in society. Policy impact, then, is also societal impact. While we have studied the impact on policy, there is also the impact of policy as the impact aftermath or extended way in which universities matter. None of this happens simply by chance; on the contrary, there is ample agency in matters. Policy impact—and the production, brokering, and uptake of knowledge on which it is founded—is the outcome of a collaborative process that requires agency on the part of all involved. As we hope to have argued convincingly and demonstrated empirically, exploring impact through governmental reports provides an effective approach to understanding these dynamics further, particularly as it allows for a view of the text as brokering interaction between researchers and stakeholders. In science–policy interaction, a space between science and policy is waiting to be filled with knowledge-brokering texts. Herein lies an invitation for universities to matter more.