Introduction

Technical standards are becoming increasingly important in contemporary economies and societies, in particular due to the further expanding role of information and communication technologies (ICT). These technologies rely heavily on standards in order to realize the interoperability of an increasing number of components and to exploit positive network externalities (David & Greenstein, 1990). Complexity in other technological areas and industries has also risen significantly. In addition, standards are used more and more as instruments of co- or self-regulation to support governmental regulation. This is particularly evident in the European Union, in which the New Approach, later expanded to the New Legislative Framework, connects European directives to harmonized European standards (Borraz, 2007).Footnote 1 Here, standards are an instrument to specify general regulatory requirements related to health, environmental and safety issues. Overall, the increasing relevance of standards to all areas of the economy and, therefore, society has been recently acknowledged, e.g. by their inclusion in the OECD Oslo Manual about measuring innovation (OECD/Eurostat, 2018). In contrast, research and development in science and technology is already an established area of research since several decades as it has become clear that economic growth and international competitiveness are increasingly driven by innovation. What has been much less studied in general to date are the links between science and research on the one hand and standardization on the other hand. In particular, scientific publications, which are referenced in standards, have not been in the focus of researchers. However, these so called standard-relevant publications provide—in contrast to the narrow focus of standard-essential patents on ICT technologies—an evidence-base for the link between research and standardization for a rather broad portfolio of scientific areas or technologies. These scientific publications can be the result of both applied and basic research. Therefore, it is possible to investigate how the knowledge flow from research to standardization is organized in a large portfolio of scientific and technological fields, in particular also from basic research, via standard-relevant publications. In the reference list of standards, they can be also considered as informative references in contrast to the much larger number of normative references to other standards (Fenton et al., 2018) characterizing the knowledge links but also the coherence within the whole stock of standards. The normative references have been only recently addressed by Baron and Spulber (2018) in their analysis of 36 standard setting organizations (SSOs) relying on the metadata included in the database PERINORM complemented by data taking from SSOs’ websites, but have been already in the focus by Gamber et al. (2008) investigating just standards released by the German standardization body DIN.

We take the framework of knowledge utilization (e.g. Landry et al., 2001; Weiss, 1979) to position standard-relevant publications in the context of studies on evidence-based research. A few empirical studies have connected scientific publications in policy-related documents and reports (Bornmann et al., 2016, Haunschild & Bornmann, 2017, Youtie, 2017). The large corpus of more than 20,000 standards released by the International Organization of Standardization (ISO) serves as starting point for the selection of interview partners for our explorative study to investigate how standard-relevant publications are developed. Since we cannot rely on previous research both related to the process of referencing scientific publications and the motivations of their authors, but also possible barriers and eventually impacts, we decided to interview highly cited or highly active authors using a semi-structured interview framework. This represents a starting point for a first investigation of this phenomenon of standard-relevant publications; conducting surveys based on a closed questionnaire might be a further next step of quantitative statistical analyses.

Therefore, our analysis on standard-relevant publications expands the empirical analyses on science-based regulation to standards as results of self-regulatory processes, but also complements the large body of research on standard-essential patents. The findings help us to position standard-relevant publications in the context of the different models of knowledge utilization.

The remainder of the paper is structured as follows. In the following literature review, we put publications referenced in standards into the context of knowledge utilization models, before we derive our research questions, which guide our explorative empirical analysis. After presenting the methodology including the empirical background by giving a general descriptive overview of standard-relevant publications referenced in ISO standards in the fourth section, the results of the interviews are presented in the fifth chapter, before we discuss them in the sixth chapter and derive in the final part the implications, but also point to the limitations of our approach and options for future research.

Standard-relevant publications in the context of knowledge utilization models

Since the concept of standard-relevant publications is a new one, we searched for an appropriate conceptual framework. This search drew us to the body of literature on knowledge utilization models, which complements the literature on knowledge and technology transfer.

An analysis of how scientific publications are referenced in standards can be framed in the general work by Sabatier (1978) about the utilization of technical information by administrative agencies. According to Sabatier (1978), the amount and type of technical information presented to decision-makers in administrative agencies in a first step are affected by: (1) the resources available, (2) the characteristics of the activity or issue being discussed, (3) the legal and political context, and (4) the anticipated reaction of the decision-makers. In a second step, the influence of technical information on policy decisions is according to Sabatier (1978) influenced by: (1) the resources of the information source, (2) the content of the message, (3) the timeliness of the message, (4) the political and policy context, and (5) the resources and perspectives of the decision-maker. However, he considers in his conceptual framework neither standardization as kind of self-regulation nor the direct involvement of scientists in drafting regulations. Complementary to Sabatier (1978), Weiss (1979) developed a typology of seven different meanings of “knowledge use”. More recently, however, Landry et al. (2001), Contantriopolous et al. (2010) and others have had a focus mainly only on a subset of these. For analyzing standard-relevant publications, we follow Contantriopolous et al. (2010) and rely on the knowledge-driven or science push model, the problem-driven or demand pull model, interaction, and the political model.

The knowledge-driven or science push model (Landry et al., 2001) rests on the idea that research discloses findings, the major determinant of knowledge utilization, that may be relevant to public policy and that procedures should be implemented to ensure that promising knowledge is tested and, when pertinent, implemented by decision makers and practitioners. Based on the survey by Landry et al. (2001), content attributes, notably, efficiency, compatibility, complexity, observability, trialability, validity, reliability, divisibility, applicability and radicalness influence research utilization on the one hand. On the other hand, the type of research, i.e. basic, applied, general abstract, quantitative, qualitative, particular, concrete research domains and disciplines are relevant. These insights complement the findings of Perkmann et al., (2013, 2021) on determinants of knowledge and technology transfer structured into demographic, institutional and organizational factors.

Since the transfer of knowledge to users is without explicit responsibilities, incentives are not automatic, and raw research results are not directly usable knowledge, demand pull (Landry et al., 2001) or problem-solving models of knowledge utilization have been developed. These models reverse the logic of science-push models, claiming that research should try to provide empirical evidence that helps solve pre-identified policy problems; users are thus the major source of ideas for directing research. In the demand-pull model, knowledge utilization is explained by the needs of the users, i.e. the use of knowledge is increased when researchers focus on the needs of users instead of focusing them on the advancement of scientific knowledge.

The problem-driven model falls short in that it does not consider that even research which produces insights to solve problems may be ignored because it may be not in line with the organizational interests of the users. Research results are more likely to be used when they support the interests and the goals of the organization. Furthermore, the model can be criticized for its focus on the interests of the users and for neglecting both that different types of knowledge lead to different uses and that the interaction between producers and users of research findings may influence knowledge utilization.

The interaction model has been developed to address the critique related to the supply and demand driven models. It consequently claims that those engaged in policymaking seek information from a variety of sources through a disorderly set of interconnections that eventually contribute to making sense of the problem and developing solutions.

The interaction model assumes that the difference between the culture of science and the culture of users leads to a lack of communication and, consequently to low levels of knowledge utilization (Oh & Rich, 1996). In addition, the interaction model predicts that with more sustained and intense interaction between researchers and users, the utilization of research results is going to increase. Unlike the science-push and the demand-pull models, the interaction models focus in particular on the relationships between researchers and users at the different stages of knowledge production, dissemination and utilization.

Finally, in the political model, users perceive that research is not likely to change according to their needs. Consequently, those research results become ammunition for users that find their conclusions congenial and supportive for their personal or organizational interests (Contantriopolous et al., 2010). In this model, there exists a high level of polarization of the content and division of efforts of knowledge exchange strategies between the users and producers as two dimensions.

Complementary to the different conceptual models of knowledge utilization, the empirical measurement of knowledge utilization is a significant challenge. According to the review in Landry et al. (2001), studies are focused on the instrumental use of knowledge, i.e. the contributions of knowledge to the decisions of practitioners and decision makers. A small number of studies have added conceptual and symbolic uses of knowledge as the non-instrumental aspects of utilization. Instrumental use describes cases where the knowledge of a single study induces users to make decisions that would not have been made otherwise. As for conceptual use, it refers to cases where knowledge provides new ideas, new theories and new hypotheses conducting to new interpretations about the issues and the facts surrounding the decision making contexts without leading decisions to be changed. Finally, symbolic use of knowledge takes place when practitioners and decision makers use knowledge to legitimate their own views. In contrast to the focus of studies on instrumental use, Weiss (1980) claimed that instrumental use is rare and when observed, it would tend to be more frequent in private than in public organizations, whereas symbolic use is more prevalent than the two other modes of use. Meanwhile Amara et al. (2004) have shown that all three types of use of research simultaneously play a significant role in government agencies with large differences between policy domains driven by a small number of significant determinants.

Though examining utilization with this typology may be helpful to provide insights into knowledge utilization, it may not completely address the above elaborated conceptual complexity of knowledge utilization. Consequently, addressing it in terms of only three dimensions is likely to produce an incomplete picture.

Knott and Wildavsky (1980) distinguish seven levels of knowledge utilization including reception, cognition, reference, effort, adoption, implementation and impact. Based on Knott and Wildavsky (1980), Landry et al. (2001) elaborate a model of knowledge utilization, which contains six levels (see Table 1) for their empirical study. It shows that the most important determinants of knowledge utilization are mechanisms, which link the researchers to the users. In particular, the dissemination efforts and the adaptation of research outputs undertaken by the researchers, but also users’ context and the publication assets of the researchers are considered. Researchers’ experiences complement those made by governmental officials, that users’ adaptation of research results, but also their acquisition efforts and their organizational contexts and links between researchers and users predict very well the interest of research by government officials (Landry et al., 2003).

Table 1 Stages of knowledge utilization
Full size table

In the context of stage model developed by Landry et al. (2001), Stage 3, referencing of scientific work in documents developed by practitioners and professionals is mentioned. In contrast to some surveys among and researchers or governmental officers mentioned above, there are so far only very few studies, which investigate these references in a quantitative manner. According to Haunschild and Bornmann (2017) relying on more than 100 policy-related sources, less than 0.5% of papers listed in the Web of Science are mentioned at least once in policy-related documents, in the area of climate change, the share is slightly above 1% (Bornmann et al., 2016). In a much more limited study looking at 80 government documents published by the Australian government, Vilkins and Grant (2017) identify slightly more than 4,000 references, whereof, less than 2,000 are from peer-reviewed journals. Youtie et al. (2017) and Bozeman et al. (2019) analyze the information base of National Research Council reports in order to identify internal and external dimensions of credibility, i.e. the believability of information, and its relevance in policy making processes. Overall, the empirical research on science-based policy making is still in a rather emerging phase.

It is within this research on scientific documents referenced in policy documents and reports, i.e. the third stage in the model of Landry et al. (2001), that we can locate the referencing of scientific articles in standards, the so called standard-relevant publications. We identify them as standard-relevant publications, if they are listed in the reference list of a standard, because they are the disclosed scientific evidence base of a standard. They are not, however, necessarily essential for its implementation, as with standard-essential patents. In theory, there might be both further relevant but not referenced and “negative” citations, which narrow the focus of a standard but are not necessary for its operation. However, standard-setting organizations are interested in limiting the length of standards and are in general therefore not interested in including negative or otherwise unnecessary citations.

The standards issued by national, supranational and international standards bodies exhibit a distinctive collection of features in comparison to other forms of social regulation and the coordination of expectations of behavior between actors. These distinctive features provide the starting point for the analysis presented here. Unlike what might loosely be termed social norms, standards are explicit and formalized descriptions of the processes, objects or vocabularies they seek to standardize. Further, where social norms may be known or acknowledged to different degrees by different social groups, standards claim to have universal applicability within their geographical and substantive scope of validity.

As is often observed, many standards are purely or largely conventional in their content. It is not objectively better to use specific colors to denote the earth, live and neutral wires in a plug, or, as in a frequently cited example, to drive on the left side of the road or on the right. Standards, which specify vocabularies for talking about objects, processes or materials in a particular field, exemplify such conventionality. In all of these cases, whilst the content is conventional, the benefits of all actors involved in the field, be they drivers or manufacturers, adhering to the standard are obvious.

However, other standards set out a method of doing something that it is claimed to be objectively good, where “good” may variously mean effective, safe, reliable or efficient. The International Organization for Standardization (ISO), for example, describes itself as “creat[ing] documents that provide requirements, specifications, guidelines or characteristics that can be used consistently to ensure that materials, products, processes and services are fit for their purpose”. That a particular method has these properties, and thus merits adoption as a standard, is a claim that may well emerge from scientific research and is capable of scientific verification and testing.

It is thus understandable that the formalized texts of some standards may refer directly to the results of scientific research by citing them. Given the consensual and voluntary character of standards, one of way of seeing such citations is as part of legitimating them in the sense of instrumental or even symbolic use of knowledge (Weiss 1980, Rich 1991), both in the initial process of development and agreement of a standard and in the subsequent period of its dissemination and adoption (Botzem & Dobusch, 2012; Brunsson & Jacobsson, 2002, p. 24). Some kinds of standard must at least appear to have scientific support, irrespective of the actual consensus of academic opinion in the field (Timmermans & Epstein, 2010). Here it is worth being more specific about what “legitimacy” might mean, and what scientific expertise might contribute to it. Often, a distinction is made between ‘input’ and ‘output’ legitimacy (Botzem & Dobusch, 2012, p. 741). By the former is meant the legitimacy lent by the visible participation of those who are to be subject to a standard in its formulation; by the latter, the rationally effective outcome of a standardization process. Put another way, the distinction is between the legitimacy of the procedure of standardization and the functional capacity of the resulting standard. The use of scientific knowledge might conceivably serve both ends, although in different ways. Scientific results might serve output legitimacy in the production of rationally more effective standards, which corresponds to the instrumental use of knowledge; alternately, they may support the reaching of procedurally legitimate decisions by winning the support of the actors who are to be bound by the agreed standard; “standard setters may generate input legitimacy by strategically engaging third parties [such as scientists]” (Botzem & Dobusch, 2012, p. 741).

There are numerous ways in which scientific expertise is engaged in standard-setting: through the formal participation of scientists in committees, informal advice and consultation, collaboration between individual standard-setters and scientists, as well as reading of the scientific literature (e.g. see Stage 1 in Table 1), which may or may not be formally acknowledged by a reference. However, since we are largely unable to directly observe the negotiation of standards in the closed committee meetings or informal consulting, using the formalized texts of standards including the referenced authors is one way to attempt to study this legitimation. We can assume that explicitly citing a scientific work in the text of a standard is intended by its authors in the sense both of the instrumental or symbolic use of knowledge utilization to produce an effect, even if we cannot assume what precisely that intention is.

The connections specifically between standardization and science have been investigated from the point of the view of the scientific literature. Choi et al. (2012), for example, present a bibliometric analysis of papers drawn from the Web of Science database to demonstrate both the growth of scientific literature on standardization and its concentration in specific scientific subject areas. Gamber et al. (2008), conversely, work from standards documents themselves, using the corpus of German DIN standards. From an analysis of cross-references of these documents across substantive fields of standardization (according to the International Classification of Standards, ICS) they argue for the existence of clusters of activity with demonstrable spill-over effects (Gamber et al., 2008).

Since we are still very much in the explorative phase of this research, we propose no hypotheses, but derive in the next section research questions how the standardization process integrates scientific publications, what are the drivers in particular for the authors of these papers and eventually what are the impacts.

Research questions

International standards and the included standard-relevant publications are not limited to ICT, but appear in different areas of standardization activity. Consequently, the publications include not only the results of applied research, but cover also areas of basic research and are authored both by actors conducting applied research, like companies and research institutes whose remit includes standardization, but also by researchers in universities and research institutes, where basic research is carried out.

Most standardization bodies have a formal, often web-based declaration procedure, which allows owners to declare those patents in their portfolio to be essential for the implementation of standards. Although owners are encouraged to make these declarations in advance of the final publication of the standard, some patents are also declared later. By contrast, the bibliographic reference list of a standard is published at the moment of the release of the standard. This means that all scientific publications have not only to be published, but also to be integrated into the standards before this point. Since the evolution of the composition of the reference list of standards has not yet been studied, a first research question related to the standardization process is:

RQ1:

How are the evolution and finalization of the reference lists of standards organized?

Referencing scientific publications in standards is not driven by immediate commercial incentives, because there are no proprietary rights connected to them that can be exploited by the authors. Consequently, Blind et al. (2018) have already shown that researchers active in standardization are primarily intrinsically motivated. In contrast, patenting researchers are also driven by financial interests, whereas those researchers focusing on scientific publications are typically primarily interested in their reputation among their scientific peers (Blind et al., 2018) reducing their likelihood of being actively involved in standardization (Zi & Blind, 2015). It remains to be researched whether authors of standard-relevant papers are more intrinsically or reputation-driven or if even further motivations, e.g. influenced by their research organisations, play an important role. Therefore, the second central research question is:

RQ2:

What are the drivers for researchers or organizations to get their scientific publications referenced in standards?

A further area that we investigate is the impact of standard-relevant publications. Related to standard-relevant publications Raven and Blind (2017) find focusing on a consortia developing standards in biotechnology that higher cited publications are more likely to be listed in the reference lists of standards and after the release of the standards the referenced publications receive more citations for a longer period of time compared to a control group. More recently, Fenton et al. (2018) looked at the long-term citation patterns of scientific publications referenced in standards and find that they are more highly cited than comparable scientific papers in their scientific field. However, this last paper did not consider whether this high citation rate is an effect of being referenced by a standard, or whether rather it is simply that standards tend to refer to papers which are in any case prominent within their own scientific field. In our analysis, however, we are interested to cover a broader set of dimensions of impact, beyond the mentioned citation patterns. Therefore, the third research question is:

RQ3:

What are the impacts for researchers of being cited in the reference list of international standards?

Underlying these research questions are factors (see also Perkmann et al., 2013, 2021) through which we explain the involvement of external academic researchers in standardization, which can also attributed under the knowledge-driven or science push model of knowledge utilization (see above e.g. Landry et al. (2001) and Contantriopolous et al. (2010)). We conceive of there being three broad types of factors, which encourage or discourage such engagement:

  • individual factors at the level of single researchers, including the specific topic of their research, personal interest in standardization, and the stage and trajectory of their career;

  • organizational factors arising from the mission of the employer for which the individual researcher works;

  • institutional factors, such as the salience of standardization in different scientific disciplines and their relevance to regulation and public policy.

All of these factors are mediated by any barriers or costs which involvement in standardization may entail. Finally, the outcome of the standardization process is both the standard itself, as an output, and any further impacts for the researcher, such as (potentially) enhanced scientific reputation or increased citation of their publications.

Methodology

Our initial research examined a database consisting of roughly a quarter of a million individual references from all ISO standards published by ISO by the end 2017 (see Fenton et al., 2018 for more details). For each reference, the dataset contains the reference line from the bibliography as well as information about the standard which contains the reference, such as its identifier, publication date and the International Classification for Standards (ICS) subgroup or groups under which it classified. Around three-quarters of all of these references are normative cross-references to other ISO standards (see Baron & Spulber, 2018). However, our focus is on the remaining references which are designated "mixed citations" and which appear in the "bibliography" section of standards. These references are a variety of documents, including scientific articles and conference proceeding papers, PhD theses, government reports, standards issued by national institutes, and statutory regulations. Furthermore, a variety of citation formats have been used in presenting these references.

In order to extract key pieces of information, such as author, title and year of publication as well, where applicable, journal name or publisher, Fenton et al. (2018) use an open-source citation parser to classify the different kinds of publications. Finally, their matching process identified 13,162 matches from the references to Web of Science entries, covering 10,843 distinct scientific publications. The ICS groups "Medical Equipment" and "Nanotechnologies" each alone account for around 1,000 of these papers, and further groups (of the 392 in total) likewise account for a large number of papers both in absolute terms and relative to the number of standards issued in them (Table 2).

Table 2 The most common International Classification of Standards (ICS) groups of ISO standards citing papers within WoS. Counts are absolute number of scientific papers that are cited.
Full size table

Complementary to the list of ICS fields referencing a high number of scientific publications, other fields are characterized by scant reference to scientific results, suggesting a high degree of conventionality in the standards; to this set belong, among others, road and vehicle engineering, packaging, distribution, textiles and clothing as well as generalities and management. A second set cites scientific papers extensively, and in so doing makes use of publications of above-average recency, suggesting that the translation of results into products and processes is concentrated in activities belonging to these fields. In the second group are found mathematics and natural science—the field which includes the nanotechnologies group—as well as health, environmental and food technology. A third group exhibits a modest degree of scientisation and refers to relatively old and established scientific papers. This more "conservative" group includes fields such paper, glass and ceramics, construction materials, petroleum and civil engineering.

The raw counts of papers just presented serve to give an impression of the intensity of use of scientific papers within different standardization fields. These counts can, however, be somewhat misleading in describing the extent of scientific references in the same fields, as a small number of standards contain very many scientific references (upwards of 50) and most of those that do cite scientific papers cite only a small number (less than ten). As a measure of the extent of scientisation, therefore, we prefer to measure the proportion of papers within different fields and at different time points, which cite any scientific papers at all. By this reckoning, in aggregate, only a quarter of standards cite any scientific papers. However, this proportion varies starkly between different fields. Here is again apparent the great extent of use of scientific papers in natural sciences, metrology and environmental standards, and the near absence of scientific references in fields such as materials and road engineering. In addition, there is both increasing use of scientific literature in fields that already made above-average use of such results, such as natural science, metrology and food technology, but also an increasing scientisation of fields that had not previously exhibited extensive use of scientific papers: services and company organization, information technology and chemicals.

The matching of references cited in ISO standards to the Web of Science database also permits us to identify the location of the scientific authors according to their institutional affiliation. This can be used to derive the country of scientific authors at the time of their paper’s publication, and in turn to give an indication of the contribution of different countries’ “national science” to ISO standards. Table 3 counts the number of author-paper contributions to scientific papers by country of author’s institutional affiliation. Where a paper has multiple authors, each author is counted once, not fractionated, and the total number of contributions is greater than the total number of papers.

Table 3 Number of author contributions to scientific papers cited in ISO standards by country of author’s institutional affiliation. Only countries with > 500 author contributions shown. Many papers have multiple authors
Full size table

The list of countries with more than 500 such contributions is dominated by established advanced industrial countries, with the USA, Britain and Germany in the top three places (see Table 3). The degree to which scientists based in different countries are cited varies according to the field of standardization concerned. American authors, for example, dominate the contributions to nanotechnology standards, whilst British scientists are highly represented in chemical analysis. Notably, the rapidly increasing contribution of Chinese authors to scientific production as well as growing Chinese engagement in ISO standardization committees is not yet reflected in the aggregate figures across the whole period.

Among these scientific publications referenced in ISO standards current in January 2018, we identified 2115 contributions by German authors to 1,158 distinct papers. The earliest of these papers date back to 1980, when our source database begins; the most recent were published in 2017. Although the Web of Science does not unambiguously identify the same authors across papers, at our best estimate these contributions came from 1501 distinct authors. Of these, 1177 were authors for a single paper only, with the rest having contributed up to 12 distinct papers, which were cited in an ISO standard.

As expected, the journal articles and papers, that are included in the Web of Science, are most commonly written by individuals who hold an academic position in a university. This is also true for the scientific papers that are referenced in ISO standards. However, individuals working in other kinds of organisations also publish in the scientific literature. In the case of Germany, this includes researchers working in member institutes of the independent research societies, such as the Max Planck Society, the Helmholtz and Leibniz Associations and the explicitly application-oriented Fraunhofer Society. It also includes researchers working in private enterprise, in not-for-profit bodies, and working for ministerial research institutes (Ressortforschung). The latter provide policy advice to federal and state ministries and provide research-based services in regulation and monitoring (Barlösius, 2008; Wernitz, 2015).

Table 4 shows the numbers and proportions of scientific papers indexed in Web of Science that are cited in ISO standards by the affiliation of the author, and compares these to all papers by German authors indexed in Web of Science in 2017. The comparison is made with 2017 publications, but the proportions of papers coming from different institution types overall has remained largely stable (Stahlschmidt et al., 2019). It has to be noted that papers in this table are counted on a fractional basis, meaning that each co-author is assigned only a fraction of the count for the paper; the figures are consequently lower than the total count of contributions cited above.

Table 4 Number of Web of Science papers written by scientists working in German organizations, by organization type, comparing papers that were cited in ISO standards versus all journal articles and proceedings papers published in 2017 and indexed in WoS
Full size table

Whilst for scientific papers in general, researchers at universities author a majority of papers coming from Germany, these authors account for less than half of the papers which are cited in ISO standards. Conversely, of the papers cited in ISO standards, a very substantial proportion comes from authors in ministerial research institutes (Ressortforschung) and in private industry—20 and 10 per cent respectively—although these form a negligible proportion of all German scientific output indexed in the Web of Science. Contributions from authors at "All Other" institutions are also more present among the papers cited in ISO standards; under this category fall non-profit organisations, smaller firms and a variety of federal and state agencies.

Our data come from 27 semi-structured interviews with researchers conducted in late 2018 and early 2019. The basis of our selection of interviewees was the preceding analysis of the German authors who have at least one scientific publication which was cited in an ISO standard and for which a matching record in the Web of Science was found.

The semi-structured interviews following the conceptual model of Perkmann et al., (2013, 2021) on university-industry relations were conducted among the identified German authors cited in ISO standards (see Table 5 for an overview of the characteristics of the interviewees). Interviewees, whose most recent scientific publication cited in a standard had appeared before 2000, were excluded, as were those no longer working as researchers; emeritus professors still actively researching were, however, included. The interviewees were selected in order to represent a spread of all the institution types listed above, and to include multiple institutions. A strict quota was not used, and university-based researchers were in any case much less likely to respond and those who did respond were less likely to agree to an interview. Within each group, contact was sought first with researchers who had a high number of referenced papers on the one hand and those authoring papers that were highly cited. Of the achieved interviews, one quarter of the interviewees were working for independent research societies and one third for ministerial research institutes; of this second group, over half were working for the Physikalisch-Technologische Bundesanstalt (PTB), reflecting the preeminent place of this institution’s research in standardization. Researchers in universities (22%) and private enterprise (18%) are less represented in our interviewee group. The interviews were carried out face-to-face or via telephone, and varied in length between 15 and 65 min with an average of 33 min.

Table 5 Interviewees by organization type and characteristics (code in brackets)
Full size table

The interviews were recorded, transcribed and then coded not based on a grounded theory approach, but according to the factors of the conceptual model of university-industry relations by Perkmann et al., (2013, 2021). Based on the codes eventually categories have been derived and summarized into a tabular format to facilitate the analysis of motivations, barriers and context. To validate our main findings they have been commented by a representative of the management board of the German Standardization Institute DIN having a long experience as secretary of national, European and international standardization committees, and a former manager working at ISO. Their comments have been integrated in the summaries of the interviews.

Results

We present the insights from the interviews commented by two standardization experts following the three research questions derived above.

The standardization process

Before we answer the first research question, it has to be mentioned that ISO has published rules on referencing documents distinguishing between normatively referenced documents published by ISO itself on the one hand and informative references to be listed in the bibliography on the other hand. The bibliography is not mandatory, but conditional dependent on the existence of informative references. It lists, without normative force, those documents, which are cited informatively in the document, as well as other information resources. We refer in our paper and the interviews to the references in the bibliography. The ISO representative stated that all ISO standards relied implicitly on external knowledge sources, but less than half of them reference peer-reviewed papers in their bibliography. Therefore, he concluded that there is no need for a formalized process or formal rules for them, because ISO and other standardization bodies do not operate with a degree of stringency related to scientific references. In his view, this reflects also that currently no need is seen in standardization bodies to introduce a concept like standard-relevant publications. In addition, standards have typically more pragmatic objectives, which do not need explicit scientific references.

Addressing the first questions about the development and finalization of the reference lists of standards organized, we find the following main patterns. Only in less than 10% of the cases, particularly of very well known top researchers, are scientific publications referenced in standards without the authors or co-authors being themselves actively involved in standardization. However, the involvement in standardization was sometimes not a strategic decision, as (1) states: “At some point I got into it by chance.” In other cases, experts “have approached standard setting bodies directly by themselves or via partners in research projects” (20).

This pattern is, therefore, quite similar to standard-essential patents (Kang & Motohashi, 2015), where the presence of the inventor in standardization meetings is significantly increasing the likelihood of their patent to be declared as essential. In general, a long-lasting involvement, in particular on the international level, but also at national and European standardization is necessary to successfully position one’s own research results as the basis for new standards. However, in contrast to standard-essential patents, authors of important publications relevant to the field of standardization “have been several times invited” (12) by standardization committees in order to link standards to the relevant scientific literature and increase consequently the input legitimacy.

In general, there is according to ISO and several interviewees no “formalized process for the screening and selection of scientific literature” (15), which is going to be referenced in standards. However, (2) states “of course we do a literature search before we start”.

In the case of patents, standardization organizations also do not conduct a screening of the patent landscape, but rather ask the involved participants to do so, at least for their own patent portfolio. In contrast to standard-essential patents, outsiders have no formalized option to “declare” specific scientific papers to be relevant for the content of a standard and to be mentioned in the reference list. Therefore, the experts directly involved in the standardization process have an advantage in being able to propose their topics and research results to be considered in the standardization process. Eventually, the references are—as pointed out by the ISO representative—subject to a collective decision process by working groups, technical committees and the ISO members and not based simply on the declaration by individual organizations or researchers. However, there is according to the ISO guidelines in general also the aim of limiting the number of the references listed in a standard, because this has implications for its total length and therefore also for its production costs and eventually price.

Overall, the “composition of technical committees or working groups is decisive for the screening and eventually referencing of scientific literature” (18), i.e. committees without a significant participation of scientists have neither a specific interest nor often sufficient capacity to perform these tasks. This composition of technical committees, and especially the proportion of members who are themselves scientists, varies greatly between different fields of standardization.

We now turn to the linkages between the research field and standardization in general, different motives of the author´s organization for participating in the standardization process as well as the profile and professional background of the selected interview partners and corresponding motives and barriers to actively promote standardization.

Influencing factors

In line with Perkmann et al., (2013, 2021), the answers related to the influencing factors for participation in standardization addressing the second research question have been grouped into institutional, organizational as well as individual factors.

Institutional factors

The authors were asked for general reasons for the publication of research findings and on which journals they are focusing. For scientists working at research institutes and universities it is one of the core goals to publish their work in order to increase their visibility and reputation within a certain research field. In very few areas, it is prestigious, “like a research prize, if a standard setting body with high international reputation adopts a standard, where you have contributed to” (6).

For authors affiliated to ministerial research institutes, they aim to have their findings widely accessible for the public and see their publication as an overarching interest of the society. They mainly focus on high-ranked journals to demonstrate the quality of their work, but the researchers focusing on more applied research also publish in more generic newspapers and booklets to inform a broader group of people outside of the research community. Authors with an industry background working at a company have no strong interest in publishing their work in general, but have been mostly being members of research projects together with universities and other research organizations. In these cases, they acted as co-authors supporting the interest of university staff in publishing papers, which are often also the deliverables expected by external research funders, such as, in Germany, government ministries.

Looking at the research background of the interviewees, the findings reveal a rather stronger orientation to applied research, whereas only a few authors were focused on basic research (see Table 5). Additionally, a smaller proportion are doing both, spanning basic and applied research, albeit typically with a higher weighting to the latter. The areas of research included environmental protection, toxicology, material science, health as well as instruments and measurement. This finding is in line with existing results from quantitative analysis of the reference lists of standards (see above and Fenton et al., 2018). As pointed out by the ISO representative, in principle, all standards have external sources, i.e. even if not listed in the bibliography, standard-relevant publications could be identified for all standards independent of their subject field. All of the topics strongly involve method-based analyses, validation, legal metrology and test systems, which require standardized procedures. In order to come up with a comparable and scalable method, the scientists are often focused, for example, on the transfer of a new measurement technology into a common standard. The interviewees mentioned that they would like to close the gap in the analytical field, which thus requires a standardized method. In this context, specifically basic research creates the basis for further methodological developments and makes the scientific work comparable.

Coming up with the information about standards referring to the authors’ publication only two out of 27 scientists did not know that their publication was cited in an ISO standard. Besides, none of the authors has been aware of the higher citation rate of their papers compared to a control group of publications in the same journal and year, which had not been in the standard reference list.

Patent applications in the context of publication and standardization are not applicable and most of the contacted researchers have not heard about standard-essential patents yet. This was despite the fact that more than three quarters of the interviewees either hold a patent themselves or through an affiliated organization. However, in those cases the following line of reasoning was mentioned several times: “If you develop a new method that you would like to have widely used and then go into standardisation, then that was always more important to us than getting a patent on it.”(23).

Organizational factors

Along with the interests of the individual researcher and the previous mentioned linkages between the research area and standardization, organizations have different motives to push standardization.Footnote 2 For ministerial research institutes, standardization is often an explicitly stated part of their mission, e.g. they have “a certain responsibility as an authority for metrology” (17), or “work together with regulatory bodies to advise them” (7) in setting new or changing existing standards to fulfill their special societal purpose of providing and establishing reliable methods and approaches. It is also mentioned that “it might also be good for their business, if [they] now also have an ISO standard where the reference material is standardized” (19). Finally, even some mission statements of companies includes standardization and is for example “about standardizing the pre-analytical workflows in order to improve sample quality for analysis” (24). For companies mainly focusing on applied research, standardization is also an asset and works as a marketing instrument and certification or proof of quality.

Especially enterprises in the area of health, material science and metrology have a strong incentive to develop methods, e.g. (27) was “significantly involved in this new development or further development of measurement technology … [eventually].. interested in measuring techniques from different manufacturers delivering comparable results”, and approaches, which become a standard and describe the state of the art. Having representatives sent to standardization committee was said to strengthen a company’s position within the industry sector, and provided the opportunity for an ongoing exchange with other players within the industry, be they companies, universities or public bodies.

Universities and independent research societies identify standardization as one driver of innovation and technological progress. They are willing to enhance national and international standardization activities and promote it along research, industrial applications and politics. Together with research institutions working in standardization-affine fields, such as environmental protection, toxicology, material science, health as well as instruments and measurement, individual researchers at universities support standardization with their research findings, their active participation at standardization bodies and in promoting its importance in setting regulative framework conditions.

Individual factors

Analyzing the professional background of the interviewed researchers, they all looked back on a long and successful career, which corresponds to inventors owning patents and involved in standardization as being older than the average inventor (Ploschka 2016). Regardless of their employment, the majority has been working for the same organization or company for the last 10 to 20 years and had none or few changes of job after academic qualification. More than 40% had completed their doctoral degree or habilitation theses. Of these 72% were working in a physics-related research field. The large majority of the 27 researchers are male; only two are female. All researchers are working in a wider network and have been members of several scientific associations and working groups within international organizations.

In most cases, where scientists are not working for a private enterprise, they do not have direct industry connections and have not worked for a company during their past career. Nevertheless, in those cases, where research is partly financed via third-party funded projects, industry partnerships are in some cases established. Besides, linkages to industry are mainly established through contract research with the affiliated organization.

Independent of organizational affiliations, all interviewed researchers have meanwhile achieved leadership responsibilities, either as head of division or department, university professor or company director (see Table 5). Consequently, the higher the management level the greater the share of administrative and coordination tasks they have. The work distribution has also an effect on the activities within the standardization process and whether the researcher is active in standardization committees or not.

Motives for involvement

Following Blind and Gauch (2009), who identified the motives of researchers to join standardization in the area of nanotechnology, and Blind et al. (2018), who focus on a ministerial research institute, the interviewees were also asked for their motivations to be involved in standardization. Authors mentioned that they are active in standardization as either being a permanent member of a committee at the national or international standardization body, or as a participant during the term of a research project. In the case that researchers have not been active in the process themselves, colleagues in charge of standardization matters at the organization have been sent to the committees. In addition, some authors have been acting as experts for committees as their input was needed in order to increase their input legitimacy. In general, it observed that “now standards are coming into the field much earlier, and especially in the life sciences, the methods change relatively quickly, so that the standards also accompany this entire development process” (4).

Depending on the scientist´s organization, standardization is one of the core tasks set by their institution and is thus more strongly supported. Ministerial research institutes, for example, compensate travel expenses related to a participation in the committee and it is possible to fulfill standardization-related tasks during regular working hours, e.g. the organization “fully accepts that this is done within official time” (14). Some authors affiliated to ministerial research institutes named standardization activities as one of their main duties.

Networking effects, e.g. “networking with other testing facilities and manufacturers” (10), the possibility for “cooperation with other researchers” (21) and knowledge exchange are additional factors that positively influence the willingness to participate. For example, (5) states “I learn a lot in these working groups every time. That's because things are always discussed from very different directions, e.g. representatives from industry, government organizations and research”.

As mentioned in the previous section of linkages between the research field and standardization, representatives from the areas of environmental protection, health and toxicology contribute to societal concerns by indirectly influencing regulatory framework conditions, e.g. (22) states: “If we see gaps in the analytical field, we try to close these gaps, possibly conduct larger series of investigations. The results are then summarized and standardized.” In other fields such as metrology and testing, e.g. (7) states “we can use the test, it needs to be standardized”, standardization was described as being essential in order to compare research results and is thus needed to further develop the own research work.

Researchers within the healthcare system had developed innovative methods, which have not been used in the industry sector before, and the reason why they were willing to be active in standardization was their motivation to change the analytical understanding and to set a new standard based on the “new methods or further developments of measurement technology” (27), they had developed. For example, (4) states “My motivation, especially in preanalytics, is that once the standards have been published and are becoming more and more common in clinical practice, accreditation authorities will take them on board and accredit laboratories that adhere to these standards. And as a result, patient care will be a little better.” In some areas, the involvement in standardisation is linked to the objective to get certified, e.g. “we are, as far as I know, the first large institute that has ever received such a certification, in order to establish quality and comparability here again in the long term” (26).

Finally, standardization was mentioned as an opportunity for increasing the visibility of one’s own research and provided the possibility of transferring research findings and scientific know-how into practice, e.g. (10) has “really disclosed … in-house procedure. And that has now already become the basis for the standard” and (16) wanted “to generate impact by ensuring that the research results flow into standardisation”.

In addition to their signalling function, standard-essential patents are in contrast to standard-relevant publications much more important for companies’ immediate commercial objectives, like generating licensing revenues also via joining patent pools, but also facilitating the own market entry (Blind et al., 2011). Indirectly, both standard-essential patents and standard-relevant publications can contribute to companies’ success by securing their freedom to operate and reducing their risk of being accused of infringing others patent rights.

Again, the ISO representative stressed the fact that despite the individual or institutional incentives the decision of including or not certain references is subject to a collective process by the working groups, committees and the ISO members.

Barriers for involvement

While the vast majority of interviewed researchers knew about the referencing of their publication within a specific ISO standard, only around 60% was a permanent member actively involved in the standardization process. Stated reasons why they were not strongly involved themselves was the fact that they did not identify the participation as an important point of their present work and thus put the focus on other areas of their research; this confirms Blind et al. (2018). Moreover, the lack of time due to other administrative tasks was identified as a barrier. The voluntary nature of the committee work only “in an honorary capacity” (14) and high opportunity costs as well as the absence of financial support, e.g. according to (11) “the only disadvantage is that nothing is paid. That means you have to finance it yourself” or (7) “round robin test, that are actually needed for proper standardisation, but which no one finances” hinder a stronger involvement for those who did not have institutional funding for participation. However, some experts are not very enthusiastic: “This standardisation work, it's somehow quite boring. Mostly it is a discussion, as I know it, about the emperor's beard and splitting hairs and that is absolutely not my job" (25) or “I see it as subordinate work, i.e. it's subsequent to the scientific results and scientific studies” (8). Other researchers mentioned that the standardization process itself is a slow, i.e. “lengthy process” (6) and sometimes tortuous process due to the variety of points of view and intentions, especially in working groups with a high diversity ranging from research institutes and universities on the one hand and industry representatives on the other side and even “there is always a bit of politics involved” (6).

The researchers interviewed commonly expressed the view that standardization has a low recognition among the research community, e.g. “in research, unfortunately, standards play only a subordinate role or no role at all” (4), and little weight when scientific performance and impact are evaluated (see also the comments above). Here, the absence of authorship attribution for standards aggravates the absence of recognition, i.e. missing incentives to contribute have “to do with the fact the scientists are not quoted accordingly in the standards” (8), a problem, which was previously less severe according to Blind and Gauch (2009). This perception further correlates with the statements that a weak scientific representation compared to strong industry presence leads to discussions “no longer primarily about science but about industrial interests” (13) in some committees and prevents eventually the “best science based” solution.

Impacts

Whereas there is already evidence about the economic impacts of standard-essential patents, the impact of standard-relevant publications has only been identified in terms of long-term above-average citation rates by Fenton et al. (2018). This reflects the general fact that the commercial impact of scientific publications on companies’ performance has only recently investigated by Simeth and Cincera (2016) and Pellens and Della Malva (2018). Although, it is obviously positive despite the absence of an appropriation opportunity, Arora et al. (2018) claim to observe declining publications by company scientists over time in a range of industries and a decreasing value attributable to scientific research measured by publications, whereas the value, which can attributed to technical knowledge as measured by patents, has remained stable. Since we observe in absolute terms very few companies having published standard-relevant publications, measuring their commercial impact in quantitative terms is difficult. However, the interviewees, in particular working for small private laboratories, confirm the relevance of the company of being involved in standardization and even cited in the standards for their long-term commercial success. Via standards, the content of the research “becomes much more known in the areas of application” as stated by a researcher from an university (3).

Therefore, the focus is more on the relevance for the research organizations and universities, which are oriented towards their scientific impact. This is obviously feasible and positive according to Fenton et al. (2018) by comparing the citation structures of standard-relevant publications to a control group of publications. However, a general causal effect on citation rates of a publication by being referenced in a standard has not yet been demonstrated, in contrast to standard-essential patents (Rysman & Simcoe, 2008). Kim et al. (2017) even find that patents that cite standards are more likely to be cited by subsequent patents. Obviously, we have a virtuous cycle between standards and patents, whereas this is not the case for the relationship between standards and publications with the exception of the widely used international management system standards ISO 9001 on quality management and ISO 14001 on environmental management (Pohle et al., 2018).

However, the interviews highlighted additional impacts, like collaboration with other scientists, in particular in the case of international standardization activities. This cooperation is also reflected in the high share of (international) co-publications among the standard-relevant publications. This is unlike standard-essential patents, which are applied for mainly by inventors from one company. Since many different stakeholders are involved in standardization further contacts are available, e.g. with companies, which might lead to further collaborations including eventually research contracts.

Here, the impacts could be measured by the share of co-publications between authors with a company affiliation and scientists from research organizations and universities. Finally, the scientists of ministerial research institutes are not only the most active authors of standard-relevant publications due to their strong involvement in standardization, but they also support the ministries in drafting regulations and therefore they are “influencing regulation” (21). However, there might be also the request to “make sure that [a future standard] can be referenced in the law” (23), which might be perceived as a restriction,

However, since many regulations refer to standards (Hess & Blind, 2019), these researchers are in the position to contribute to public goods, like the protection of health, safety, including “product safety” (26), and the environment. In this context, the impact of standard-relevant publications can be measured via the references of the related standards in regulations and laws. In addition, a number of interviewees explicitly favoured such an expansion to evaluation measures.

Overall, the impacts of standard-relevant publications are much broader compared to the mainly commercial impacts of standard-essential patents, which might contribute, but only indirectly to public goods. However, measuring these different, often indirect and long-term impact dimensions is challenging and requires further research.

Discussion

In general, the analyses of interviews with authors of standard-relevant publications reveal that it is feasible to integrate scientific results codified in published scientific papers in national, European or international standards. Obviously, there is no formal (declaration) process. In addition, there is no systematic screening process searching for relevant publications. In contrast to the commercial opportunities of declaring patents to be essential for the implementation of standards, the integration of one’s own scientific work into standards does not provide such immediate commercial opportunities for researchers and their organizations. And due to the requirement to be present at standardization meeting positioning a paper in a standard’s reference list is costly.

One possible long-term benefit of standard-relevant publications for their authors is their increased visibility via higher citations rates. Such an effect has, however, not been demonstrated in the literature and in any case was not one that the interviewed authors were aware of. A concrete advantage is access to networks of researchers, which can lead to common co-publications with the related benefits for one’s own visibility. Due to the heterogeneous stakeholders involved in standardization, further collaboration opportunities exist, which can be realized leading eventually even to commercial success, e.g. via receiving research contracts both from the collaboration partners and third parties, like ministries. Finally, the involvement of regulatory bodies in standardization and the references of standards in legislation even provides the opportunity of indirectly improving the regulatory framework conditions through standardization improving its evidence- or science-base via standard-relevant publications.

Based on the new insights on standard-relevant publications, we integrate them in a framework developed by Contandriopoulos et al. (2010), who extended the typology by Weiss (1979) by taking the polarization of the content and division of efforts of knowledge exchange strategies between the users and producers as two dimensions into account. They claim that in less polarized contexts, knowledge use will likely resemble Weiss’s problem-driven model (1979), whereas in more polarized ones it will probably look like a political model of use (Weiss, 1979). When the knowledge users are unwilling to bear the costs of the knowledge exchange, a viable cost-sharing equilibrium exists only if others, whether producers or intermediaries, are motivated enough to disseminate information actively through lobby-like techniques. Again, this is much more likely in significantly polarized contexts, which allow the political use of knowledge (Weiss, 1979). In producer driven, minimally polarized contexts, a viable cost-sharing equilibrium is much less probable, but if it is found, use is likely to resemble Weiss’s (1979) knowledge-driven model.

Taking the insights from the interviews with the authors of standard-relevant publication, we can attribute standard-relevant publications on the one hand certainly to the knowledge driven model with the authors being the main force to position their research results in standards. On the other hand, some of the authors have reported that they have been asked for their input in more problem-driven areas, which is supported by the dominance of standard-relevant publications in health, environmental and safety related fields. A few of the authors have been cited even without being involved or aware of a standardization process. In some of the mentioned fields, tensions between the interests of consumer and environmental groups on the one side and industry representatives at the other side take place, which resembles a polarized constellation and therefore the political model. Although we have learnt that only finalized publications, sometimes published several years before the standardization processes have been started, are referenced in standards the interaction knowledge model still applies, if the referencing of the scientific papers is accompanied by the involvement or consultation of the authors within the relevant standardization processes. The phenomenon that some researchers publish also scientific papers about the contents of recently published standards, to which they have contributed to, definitively resembles the interaction knowledge utilization model.

Looking at standard-essential patents as another source for knowledge utilization, we observe some similarities, but also differences compared to standard-relevant publications. The declaration of standard-essential patents is certainly driven by its owners and can, therefore, be attributed to the knowledge-driven model of knowledge use. The phenomenon of standard-essential patents being declared only after the release of the related standards (Baron et al., 2014) and many of these patent applications being revised during the standardization process (Berger et al., 2012) supports the applicability of the interaction model of knowledge utilization, like for standard-relevant publications. Contandriopoulos et al. (2010) question viable cost-sharing equilibriums. However, the combination between the institutional recognition of formal scientific knowledge use by standardization organizations and strong individual-level incentives by researchers (Blind et al., 2018) and commercial interests of standard-essential patent owners results in a viable cost-sharing equilibria. Due to the fact that the users, i.e. the implementers of standards, are not very interested in pushing the declaration of patents being essential of standards, the problem-solving model is less applicable. Finally, the strong focus of standard-essential patents in the area of interoperability in the information and communication systems is in general characterized by competing technologies, but not by polarized positions of stakeholders representing different societal groups. However, current discussions about the challenges related security issues of the next generation of mobile communication standards 5G, which is—taking the number of declarations of standard-essential patents into account technologically—dominated by Huwaei (Pohlmann & Blind, 2017), provide first indications that standard-essential patents could also channel in the political model of knowledge use.

In summary, the models of knowledge utilization proposed by Weiss (1979) integrated in to the framework by Contandriopoulos et al. (2010) presents an interesting option to position standard-relevant publications, but also standard-essential patents within the context of knowledge utilization revealing additional similarities, but also differences between them. However, we have to mention that the perspectives of users’ have only be indirectly collected, because we have not interviewed the implementers of standards (Fig. 1)

Fig. 1
figure 1

Models of Knowledge Use in Standardization in Relation with Cost-Sharing and Polarization (based on Contandriopoulos et al., 2010)

Full size image

.

Implications and Limitations

The findings of the interviews with the authors of standard-relevant publications have implications for standard-setting organizations, researchers and research institutes, research funding organizations and ultimately for regulatory bodies.

First, standard-setting organizations might establish processes, which could be similar to the declaration of standard-essential patents, in order to reduce the barriers for researchers initially not active in standardization to suggest their research as likely being relevant for ongoing standardization processes. Such procedures might also facilitate the screening of relevant scientific insights by those being actively involved in standardization, because both the share of ISO standards with scientific references at all, but also their overall number are still rather limited. In general, the “informal use” of scientific results as input into standardisation processes should be the transformed into formal citations of scientific publications. On the one hand, the legitimacy of standards will be strengthened (Botzem & Dobusch, 2012). On the other hand, referenced researchers will receive more credits for the transfer of their scientific achievements in practice, which have been complained about by some interviewees. And finally, the appropriateness of using these references as indicator for the use of scientific knowledge will be improved.

Secondly, researchers should be made aware about the opportunity to increase the visibility of their own work via standardization, which is according to the limited number of scientific papers referenced in standards still not completely exploited as additional knowledge dissemination channel.

Thirdly, the generation of standard-relevant publications, like standard-essential patents, requires a strategic long-term participation in standardization, which has to be backed by the management of research organizations and universities. Here, the immediate cost for participation, e.g. travel costs and fees, needs to be covered in order to reduce researchers’ barriers for participation.

However, this is only a necessary, but not a sufficient condition. Most important is the change of incentive schemes increasing the relevance of participation in standardization, but in particular standard-relevant publications among other publications, for academic career decisions. Therefore, fourthly, research-funding organizations, in particular those with an interest in the successful transfer of their funded research, should consider not only publications and patents, but also standard-relevant publications for the selection of applicants and eventually the evaluation of funded research projects.

Fifthly and finally, the referencing of scientific articles in international standards is also an indication for an evidence-based science and technology policy (e.g. Bozeman et al., 2019) for the European Union due to the specification of European framework directives, like the machinery directive, via harmonized European standards (Borraz, 2007).

In summary, a rather comprehensive approach is needed to exploit the opportunities of standard-relevant publications for both a science and technology transfer channel and evidence-based science and technology policy.

Our explorative study has at least three major limitations. Referencing scientific publications in standards might underestimate the real science base of standards, because many more input from science might be used informally for their development as indicated by some interviewees. Although we have validated the statements of the interviewed experts by experts of standardization bodies, we have not considered in particular the views of industry representatives and other stakeholder groups being not authors of standard-relevant publications. Unfortunately, we are not able to identify them due to data protection restrictions. In addition, reviewing minutes of the technical committees is also only possible for their participants. The views of industry representatives, which have in particular in technical committees in fields of applied research the majority, might explain the underreferencing of scientific publication in standards, because they are interested in standards, which are easy to be implemented and have not too many pages limiting the price of standard documents. However, this underreferencing might have implications on the objectivity and therefore legitimacy, but also the usefulness of technical standards in practice. To investigate this follow-up research question the focus has to be shifted to those implementing standards in industry, but also referencing standards in governmental regulations.

Another limitation is the focus on ISO and not considering e.g. the International Electrotechnical Commission IEC. In electrotechnology, our findings might be not be leveraged due to other standardization processes and different roles of researchers and research input.

Finally, we focus on Germany, which has both a very specific landscape of research institutions and a rather developed standardization culture, which calls for comparative studies in other countries with different innovation systems. Therefore, these three aspects should be considered in future research.