Introduction

It was in 1987, when Eugene Garfield reminded the scientometrics and informetrics community of the importance of review articles for the functioning of an evolving scholarly information system (Garfield 1987). From his perspective, one of the most notable challenges for the scholarly communities to deal with was the unprecedented growth of science to which review articles could provide a solution. Not only, he argued, were review articles valuable for presenting the state of the art in a specific field, but they would also allow for identifying novel or relevant topics which can be taken up by future research. The articles of Garfield (Garfield 1987) were part of a lively debate about the characteristics and functions of review articles in science; and several features of review articles have been studied at that time (Woodward 1974; Virgo 1971; Adams 1961; Bastide et al. 1989; Bakker and Rigter 1985; van Buskirk 1984). As a result, the study of review articles in science was perceived as a relevant issue within scientometrics and informetrics.

In the years following, interest in review articles declined. Instead, the value of the original research article was emphasized which was perceived as the “gold standard” (van Raan 2004) in science. As a consequence, publication formats other than the research article received less attention as object of scholarly research in scientometrics (Jokic and Ball 2006). Moreover, research related to reviews in scientometrics emphasized methodological caveats for studying this particular genre, such as lack of consensus on definitions (van Buskirk 1984), lack of accuracy of document assignation in scholarly data bases, or lack of knowledge about usage of review articles (Bastide et al. 1989).

Yet, in this article, we argue exploring review articles allows for dealing with crucial aspects and mechanisms in the production of academic knowledge. As Ludwig Fleck (1980) suggested, the different document types in scholarly communication play important roles in what he considered the “production of facts”. At the time the book of Ludwig Fleck was being written, review articles did not exist as an established genre of scholarly communication. The social and epistemic functions Fleck established to be important for handbook contributions, however, have much in common with some of the features review articles are attributed to; as they provide means for achieving consensus, thereby stabilizing and shaping the knowledge base. Hence, the analysis of scholarly review articles may provide interesting directions for the exploration of different mechanisms in knowledge production. Review articles, we propose, can influence knowledge diffusion by shaping citation flows (Ho et al. 2017), by legitimating or justifying new and emerging fields (Blümel 2016), or, by contributing to the closing and opening of controversies in science.

Research within scientometrics and informetrics, has so far not intensely dealt with many of these topics. In other fields, however, interest in review articles has risen. We now find various works dealing with review papers which relate it to debates about the structures and patterns of scholarly communication, particularly in the field of linguistics (Azar and Hashim 2014), the sociology of science (Bastide et al. 1989), and, more recently, in biomedicine (Glasziou et al. 2014; Ioannidis et al. 2014b). Some of these works challenge existing conceptualizations of review articles in scientometrics, while others may be fruitfully combined with existing research in our field. A research agenda dealing with review articles may allow for taking up some of these ideas in order to enhance our understanding about the above mentioned processes of knowledge production.

Against that background, the aim of this article is to review recent developments for the study of review articles, to bring together different perspectives for its study, to discuss theoretical, conceptual and empirical accounts of how reviews can be understood and defined, and to identify important methodological challenges for the study of review articles as well as potential solutions thereof. As a result, the paper is aimed at providing a proper research agenda for scholarly review articles, building on perspectives from other fields and other methodologies than are currently deployed. The paper is based on work conducted within a project on the study of review articles in science and draws from the results of an international workshop conducted at the Conference of the International Society of Informetrics in Rome in September 2019.

The paper is structured as follows: In the first section, we deal with one of the most substantial challenges for studying review articles, the absence of a commonly accepted definition. We therefore provide a working definition of the review on which subsequent sections are based on. In the second section, we shortly review the study of review articles within scientometrics. Subsequently, we summarize and identify recent problems and methodological caveats for the study of review articles, while in the fifth section we provide new avenues and recent perspectives for the uses of review articles. Based on these advances, we develop a research agenda for studying review articles taking into account the relevance of this specific genre for the study of larger phenomena in science studies beyond scientometrics and informetrics.

Difficulties of defining review articles

One of the biggest challenges for studying review articles in science is that there is no commonly accepted definition of the review article (Woodward 1974; Noguchi 2006; van Buskirk 1984; Azar and Hashim 2014). Conceptions of how to understand and define review articles differ between fields and disciplines, according to different publication practices in science (Hicks 2004), which have led to differences in how research of a field is ordered or represented. Hence, there is no such thing as the review, but many different forms thereof. Practices of reviewing, summarizing or ordering research are manifold and very different among the various scholarly fields. What is more, reviewing is a pervasive practice. The characteristics established as typical of reviews may occur frequently in other document types. For example dissertations also employ a section which can be understood as a review (Boote and Beile 2005). Therefore, it may be more appropriate to speak of an ecosystem of review literature (Azar and Hashim 2014)

Moreover, scholarly databases have established their own concepts and definitions for what makes a review article (Colebunders and Rousseau 2013). But these concepts are very different from each other. While, for instance the indexing service of MEDLINE has established very concise definitions and characteristics for review articles, concepts and algorithms of identification in Web of Science for review articles appears to be different (Ketcham and Crawford 2007). In addition, as van Buskirk (1984) established in surveying MEDLINE’s user base, the readership’s conceptualizations and definitions of what counts as a review article can differ from the data base definition (van Buskirk 1984). This hints to a recurring problem in classifying genres. While researchers refer to intellectual values when defining the review genre, data providers need more distinct and technical definitions that are incorporable into formulas and algorithms. Thus, many of the studies on genre classification or quantitative aspects of the review literature often compare very different classification regimes.

It is therefore difficult to find a concise overarching definition, capable of representing the broad variety of the document type. We argue that the definition of “review article” maybe to a lesser extent established based on textual characteristics, but on the position of review articles in the scholarly literature. Review articles are reporting and making claims about primary research. Hence, they are basically providing observations and comments to existing research they can be perceived as a secondary genre of writing (Myers 1991, p. 45). Many of the textual- and meta-characteristics of review articles, which are used by the more technical database definitions, fundamentally relate to that ‘position’. Yet, we find various forms for observing or commenting on primary literature. Review articles can aim at ordering, categorizing or evaluating primary knowledge. The goal of this article is therefore to sensitize for such diversity of review literatures, their respective audiences, and the functions which are attributed to them. Given the diversity of review article conceptions and understandings, we may need field specific analyses of the production, governance, and reception of such article formats. In the next section, we will shortly review dedicated research about review articles in scientometrics.

Dealing with review articles in scientometrics

As mentioned in the introduction of this report, interest in review articles within the scientometrics community has arisen in the 1970s. Relying on the findings of Adams (1961), Judy Virgo (1971) was the first to publish on the characteristics and functions on review articles in science. She established that review articles are of tremendous importance for the scholarly information system, as they allow for keeping up with the growing information overload in science. She studied distributions of different sorts of review literature in science, of which one she termed the “controlled literature” of review articles, which consists of review articles published in annual reviews, and the “non-controlled literature”, which is review articles appearing in scholarly journals. Since then, different types of review articles have been established. There are, for instance, categorical, evaluative, descriptive, systematic, narrative review articles (Woodward 1974; Adams 1961).Footnote 1 In contrast to those earlier works, contemporary scientometricians make use of the scholarly databases to study the characteristics of review articles. With this development, the technical features of reviews became more predominant in defining review articles. But as we will see, researchers later on started to confront technical features with cognitive features, and questioned the reliability of genre definitions by the database providers.

A recurring theme within the scientometrics literature is research on citation effects of review articles (de Almeida and Guimarães 2013; Ho et al. 2017; Jokic and Ball 2006; Knottnerus and Knottnerus 2009; Liu and Kuan 2016). Several authors have revealed that review articles have longer references lists (Jokic and Ball 2006), which correlates with more citations (Ioannidis et al. 2016), making review articles potentially more significant than, for instance, research articles (Knottnerus and Knottnerus 2009; Teixeira et al. 2013). However, while there is agreement that the review article is on average, more cited than other research items, little research has been accomplished on the differences between various citation-classes related to the different document types. There is a large proportion of review articles which are not cited or subject to delayed recognition (El Aichouchi and Gorry 2018). The reasons for that are currently not explored.

A topic closely related to the latter is the role of the review article in citation networks. As Liu and Khan (2016) have established, review articles can play an important role in relating the literatures of different fields. Based on its higher citation rates, various studies show that review articles can have an effect on the citation structure of a topic (Ho et al. 2017; de Almeida and Guimarães 2013; Knottnerus and Knottnerus 2009), or shape authorities in disputes (Greenberg 2009). Although a review may not end the citation history of its included articles (Lachance et al. 2014), its impact may influence the reception of primary research (Knottnerus and Knottnerus 2009). Some authors have therefore attributed review articles biasing effects on citation figures (Moed 2005; Moed and van Leeuwen 1995; Teixeira et al. 2013). As an effect, it was pledged not to integrate review articles into scientometric studies (Ho et al. 2017).

Finally, much research has been conducted on the identification, assignation and coverage of scholarly output in scientific data bases (Donner 2017; Torres-Salinas et al. 2014; Brase et al. 2015), a topic which is particularly relevant to the study of review articles. The various providers and indexing services of scholarly databases, however, differ in how they define and identify review articles. Since 2007, for instance, Web of Science provides information about the document type for each article. In order to enable document specific queries, Web of Science uses algorithms, automatically assigning publications the document type “review article”. One of the methods Web of Science uses is to automatically assign publications the document type review article, if the manuscript contained more than 100 references. In addition, articles published in review journals, or in the review sections of journals, as well as those with titles containing the word “review” or “overview” have been also assigned as review articles (Harzing 2013; Sigogneau 2000; Ketcham and Crawford 2007). In the documentation material of the scholarly database Scopus, however, the assignation of review articles is related to the length of the reference list and article structure (Scopus 2017, p. 11). In the MEDLINE/PubMed database, document types are subject to the standardized and controlled MeSH vocabulary and assigned by human indexers, which mostly use publication type suggestions from the data providers, the scientific journals. Summarizing, the heterogeneity of assignation procedures produces various problems: As a consequence, a growing proportion of research is dealing with the problems of identification, assignment and coverage of review articles and other document types in various indices and scholarly databases (Moya-Anegón et al. 2007; Torres-Salinas et al. 2014; Donner 2017).

Reviews as avenues for research beyond scientometrics: research quality, field construction and the pervasiveness of indicators

Though research in scientometrics has dealt with methodological issues of studying review articles, the dynamics and the differentiation of review articles is not systematically accounted for. Although several studies state that the proportion of review articles is rising (Moed and van Leeuwen 1995), we find few studies exploring the causes thereof.Footnote 2 Particularly in biomedicine, a relative increase of review articles can be observed (Michon and Tummers 2009; Colebunders and Rousseau 2013; Colebunders et al. 2014; Ketcham and Crawford 2007).Footnote 3

One of the potential forces driving the attractiveness of review articles in the biomedical field is the debate about the quality of research. Since 2014, the Lancet published a series of articles about the crisis of quality in biomedical research. According to the authors, a high number of studies are perceived as waste in research because of low quality in conduct and reporting (Ioannidis et al. 2014a, b; Chalmers et al. 2014). Similar arguments are being made in the realm of psychology (Vazire 2017). As a solution to that problem, review articles, in particular, systematic review articles are perceived viable because they are expected to guide the design of further studies (Glasziou et al. 2014). In accordance, we find various authors arguing for the superiority of systematic reviews in serving the paradigm of evidence-based medicine (Grant and Booth 2009). As a consequence, the genre has become so important to the medical sciences that transnational institutions like the Cochrane Collaboration formed around the production and dissemination of systematic reviews. Even non-medical fields have adopted the idea of evidence-based practice and now leaning heavily towards the promotion of systematic reviews (Chalmers et al. 2002). New regulations and standards for writing reviews are in place, aiming at making these items more visible (Moreira 2007).

Moreover, review articles play a role in the debate about the unintended consequences of indicator driven incentive structures in science (Espeland and Stevens 2008; Steele et al. 2006). The debate has been also taken up in scientometrics (Butler 2003; Butler and Visser 2006) and led to novel regulations for dealing with evaluations and science assessments. One of the biggest forces potentially influencing editorial and publisher decisions are bibliometric indicators, such as the Journal Impact Factor (Garfield 2006). The Journal Impact Factor (JIF) is highly influential in distributing and redistributing attention to scholarly journals, as scientists are able to choose specific places for publication (Moed and van Leeuwen 1995). Scholarly journals therefore continuously report on and monitor information about their JIF. Several studies have already revealed strategies of scholarly journals aiming at influencing, hijacking or even manipulating the indicator (Falagas and Alexiou 2008). Althouse et al. (2009) have revealed field specific differences in the usage and discrimination of impact factor, implying different JIF related policies of scholarly journals (Althouse et al. 2009).

Arguably, the review article may be of strategic use for editorial strategies in light of indicator improvement (Ketcham and Crawford 2007; Michon and Tummers 2009). Generally, reviews are included as citable items in the denominator of the Impact Factor, while yielding significant contributions to the numerator, due to their high citation rates (Lei and Sun 2020). Thus, editors may favor to increase the amount of reviews in their journals (Chew et al. 2007; Falagas and Alexiou 2008). Further, review articles are often commissioned by the members of editorial boards, particularly among the annual review journals. Relating to that established practice Lievrouw (1988a, b) have argued that review articles therefore may serve the reproduction of so called “invisible colleges” (Crane 1967; Solla-Price 1963).Footnote 4 In the context of a “pervasive influence of publication metrics” (Steele et al. 2006), it seems reasonable to assume that the commissioning of reviews is of strategic consideration (Moed and van Leeuwen 1995). Up to now, however, there are no studies on editorial practices related to review articles, whilst these are only rarely related to findings of evaluative bibliometrics.

Finally, we may find novel functions of the review article beyond those established by de Solla Price and others. As Jokic and Ball (2006) have argued, we increasingly find reviews that provide overviews for a problem which are targeted at a more broadly defined audience (Jokic and Ball 2006, p. 73). Blümel (forthcoming), for instance, shows that review articles are also used to legitimate and push emerging fields, such as that of synthetic biology (Blümel forthcoming). The reasons for that may lie in the heterogeneous and currently not well understood textual structures of review articles (Noguchi 2006), which often do not follow standardized expectations, particularly among narrative reviews. Specifically, linguistic analyses imply that such attributes relate to specific patterns of argumentation, being structural features of these texts (Bazerman 1988). In particular, the field of genre studies has contributed to our understanding of document types in the scholarly discourse, their use and their changes over time. Charles Bazerman and John Swales have argued that exploration into the development of genres in science may be suitable to show the co-production communication and knowledge production regimes (Swales 1990). Myers (1991), focusing on the case of molecular biology aimed at showing that review articles’ narrative structures potentially can influence how a given field can be or is perceived.

Seldom have these arguments been taken up within scientometrics. One of the few works is Bastides, Courtials and Callons article on “the use of review articles in the analysis of a research area”, published in 1989 (Bastide et al. 1989). Focusing on the field of polymer science, they show that review articles’ textual structures can be considered “highly programmatic”: that is, they establish relationships between fields and arguments, referring to topics not hitherto established and put them into hierarchical orders. More recent analyses (Hyland 2012) of other not clearly defined scholarly genres show that rhetoric and argumentation logics of document types beyond the research article are less well understood of which the review article is one of the most visible examples. This shows that more research is needed in order to more systematically understand the document types indexed in data bases.

Studying review articles in science: a research agenda

In order to steer debates about the potential uses and avenues for research about scholarly review articles, we aim to provide a research agenda, taking into account and elaborating on methodological and conceptual problems of review articles and potential solutions thereof. The input of this agenda is based on a literature study compiled within a project on the use and reception of review articles, but is particularly informed by a scholarly workshop conducted with experts from scientometrics and informetrics in September 2019. Each of the proposed avenues for future research is closely related to one of the former identified clusters of problems or challenges for studying review articles. In the first section, we will elaborate in more detail on future research concerning the methodological problems of identifying review articles, particularly taking the problems of scholarly databases into account. In the second section of this chapter, we suggest directions for the study of different uses of review articles, dealing with the conceptual problems of review articles; while subsequently, avenues for the textual analyses of review articles are identified. In the fourth section, we deal with research questions related to the organization of review literature, such as commissioning or editing reviewing work. Finally, we propose a cluster of research questions related to the epistemic structures of a scientific field.

Methodological caveats of studying review article resulting from the usage of scholarly databases

Research in scientometrics and informetrics has contributed enormously to acknowledging the problems researchers are encountered when using scholarly databases (Gorraiz et al. 2016; Donner 2017; Harzing 2013; Rotolo and Leydesdorff 2015; Moya-Anegón et al. 2007). Hence, problems and shortcomings, inconsistencies or deficits in cleaning research information within and among databases is integral part of scientometrics and informetrics. Research on review articles is no exclusion to that. Review articles are document types that scholarly databases, based on specific algorithms and methods, assign to specific scholarly output. However, the assignment of document types is sometimes incorrect, even if the underlying conceptualizations of the data bases are employed. As Donner (2017) has shown, precision of identifying reviews lies at 87% in WoS compared with the definition of the data base.

Moreover, we can see that algorithms of scholarly data bases have changed, posing another problem for studying review articles. The algorithms for assigning document types in web of science, for instance, appear to have changed throughout before and after 2009.Footnote 5 The same may hold true for Scopus, but we currently lack more systematic research on the topic. These inconsistencies are problematic, not only because the coverage or the structure of specific document types is misrepresented, but because errors in their identification may also affect bibliometric indicators or normalization procedures (Ioannidis et al. 2016). Therefore, more studies on replication and the scaling up of document type errors are needed. In particular, more research on document assignment algorithms using reverse engineering should be conducted, particularly for review articles.

In scientometrics, it is widely held that review articles have different citation characteristics compared to other types of scholarly document types, such as research articles (Jokic and Ball 2006). Based on this argument, several authors contend that for specific purposes, review articles may not be included in bibliometric corpora (Ho et al. 2017). However, the inclusion or exclusion of review articles is not always reported in scientometrics which can lead to misunderstandings regarding the corpus construction. Documentation of the methods therefore should also include the usage and selection of specific document types and the reasons thereof. More research, however, is needed in order to understand as to why there are differences in citation structure and the (potential) effects thereof.

Studying field specific patterns of conception, reception and usage of review articles in science

It has been mentioned above that we currently lack an accepted definition of a review article. Rather, we have different conceptions and, as a consequence, different methods of identifying review articles in science, particularly relating to different algorithms of assignation in existing scholarly databases. Hence, there is no such thing as an overarching, universal conception of a review article. We argue that one potential consequence of that situation is to more systematically study the field specific reception, usage and conceptual understandings of review articles. Some fields, such as mathematics, may not use review articles at all, while in others, the writing of review articles is widely established. Hence, there may be functional substitutes of review articles in specific fields while other communities have very specific conceptions of review articles, which are established by relying on regulations, such as reporting guidelines or formatting rules (Davis and Taylor-Vaisey 1997). In fields like the humanities, however, review articles may not easily be distinguished from more theoretically oriented research articles (Guetzkow et al. 2004). This is to say that writing review articles can be perceived as part of a scholarly discourse community’s ownership (Hyland 2012), which new entrants acquire in the process of scholarly socialization, by for instance, writing their PhD thesis (Boote and Beile 2005). In linguistics, such knowledge has been perceived to be the genre knowledge of scholarly community (Berkenkotter and Huckin 1993).

On the basis of various case studies in scholarly writing, a dedicated theory of genre in scholarly communication has rethought scholarly communication forms as emerging from continued routines of interaction (Swales 1990; Swales and Naijar 1987; Hyland 2000, 2012). Most important for understanding the notion of genre in this literature is the moment of typification, which relating to Berger and Luckmann (1969), is socially constructed as its ‘provides answers to recurring situations in the process of writing’ (Berkenkotter and Huckin 1993). Thereby, scientific formats become an own social reality, which shapes and influences scholarly reception of knowledge (Bazerman 1988, p. 8). That implies that fields have cultivated specific expectations about how to write and receipt review articles.

Hence, the field specific analysis of reception and usage patterns of review articles leads to wider questions regarding the knowledge production of specific scholarly communities. What does it mean for a specific community to provide order and which forms of ordering knowledge have been established or promoted as a consequence thereof? Studies regarding the perception of review articles therefore could complement quantitative studies of reception analyses with qualitative oriented drawing on interviews from experts of the respective field.

Beyond meta data: towards content and textual analyses of review articles

A second consequence of the above mentioned problems of defining review articles in science is that we need more research studying the content of review articles. Such research designs might be also more appropriate to study and characterize the different types of review articles. Since the beginning of research in scientometrics and informetrics, many different types of review articles were established. Drawing on John Adams (1961), Judy Virgo (1971) for instance identified two different types of review articles, categorical and evaluative reviews. In 1974, Woodward differentiated review articles by distinguishing eight different types such as interpretive, speculative, critical, evaluative, state of the art, tutorial, and popular (Woodward 1974). Yet, whilst these different types of review articles may exist, it is difficult to identify them in bibliometric corpora, since there is not always an indication of the specific type neither in the title nor in the abstracts or keywords of a scholarly document. Moreover, the different types established in the literature (for example those of Woodward 1974) have not been systematically studied and compared to each other, based on, for instance, their textual structures. Hence, one needs to study the content of the full text of review articles.

In addition, standardized review formats like systematic reviews in biomedicine share attributes beyond textual structures, for example special graphs, diagrams or charts. A common review guideline generally requires a flow diagram that clarifies the different stages of study selection (Moher et al. 2009). Reviews employing quantitative meta-analysis often use forest plots to display and weight effect sizes and funnel-plots to show a potential skewness of included studies. The former has become so common for those types of reviews, that the logo of the Cochrane Collaboration roughly resembles a forest plot. Such tools support the systematic review’s functionality by ordering a huge amount of information and make its claims persuasive (Espeland and Stevens 2008).

Content analyses are also needed to understand the value of review articles and the specific technologies employed to persuade scholarly communities of a given claim. Review articles are perceived as establishing programmatic accounts, not only describing, but also influencing research lines (Bastide et al. 1989). Various studies in argumentation and rhetoric have shown that scholarly texts employ specific techniques for persuasion. Gilbert (1977), for instance, explored the ways by which references are used to persuade readers. In order to establish the specific argumentative means of citation, Cozzens studied the citation context within scientific texts (Cozzens 1985). In 2008, Bornmann and Daniel established different uses of citations in text: affirmational, assumptive, conceptual, contrastive, methodological, negating, perfunctory, and persuasive (Bornmann and Daniel 2008). We argue that review articles are a particularly rich source to study in text uses of citations and other means for persuasion. Based on some of the assumptions made above, it appears reasonable to assume that reviews employ perfunctory and persuasive uses more than research articles.

Another reason for studying the content of review articles is to explore the ways by which they order, select or identify relevant research related to the topic of the review. Review articles often establish specific criteria for the selection of research which are used to represent a specific topic or research field. Moreover, review articles increasingly contain sections about future research questions. There is some indication that these sections contribute to the shaping of topical agendas in research field (van Merkerk and Robinson 2006). Yet, we do not have analyses comparing different fields in regard to such future research sections and their impact on the respective topical landscape. Content analyses may reveal specific patterns of presenting future research questions, but may also allow for comparing proposed and existing research.

In addition, content analyses may also be used to study changes in the evolution of a genre. For instance, Charles Bazerman (1988) has shown studying the research article in Physical Review that the articles of that specific genre increasingly became related to each other by sharing similar theoretical assumptions and propositions. According to expert experience, there are first indications, that the review article in biomedicine has changed its style and structure substantially since the 1980s, which would make it necessary to study the change of the article in more detail focusing on content analysis.Footnote 6 In current analyses of review article focusing on meta-data, such changes cannot be detected. What is even more, it appears less likely that such analyses are conducted, because many of the articles cannot be retrieved by relying on existing scholarly databases such as the Web of Science.

Yet, there are also caveats and limitations for such studies: a major problem one encounters in conducting content analyses is that they most often cannot be used to study large proportions of publications. Charles Bazerman (1988), for example, studied the evolution by selecting research articles of the Physical Review in a 2 years mode of selection, leading to a rather small corpus of texts. Increasingly, however, the technologies of text mining may allow for more easily processing larger proportions of text, particularly in combination with qualitative analysis of texts. Trained algorithms would then allow for building models automatically classifying review literature based on textual characteristics. At present, the opportunities of text mining are not fully exploited for studying specific types of texts, such as review articles. Such analyses may help to get a critical look into the document types indexed in the existing scholarly data bases.

Studying the organization and infrastructures of reviews and its effects

But the writing of review articles cannot only be understood by studying the texts of review articles alone. Increasingly, novel infrastructures for the writing and organization of review articles are emerging, specifically for the specific type of systematic reviews. Particularly in the biomedical sciences, systematic reviews are considered to be at the top of the hierarchy of evidence (Atkins et al. 2004), because they are expected to rule out certain forms of bias allowing for generating generalizable results. These attributions have emerged from specific infrastructures, such as codes of conduct and reporting guidelines, aiming at structuring the text and visual appearance of systematic reviews (Chandler and Hopewell 2013). In Web of Science, Cochrane’s library appears like a usual medical journal. However, Cochrane’s systematic reviews differ a lot from systematic reviews published in classical journals. Due to their underlying methodology, Cochrane reviews provide a registered protocol, a more promotion-oriented abbreviated version published in traditional journals, an ongoing update history, and a complex bibliography, separating primary studies into included and excluded ones, as well as methodological articles and further resources (Higgins et al. 2019).Footnote 7

In order to study the dissemination and reception of review articles, we may also take such infrastructures and their performative effects on the writing of review articles into account. Cochrane reviews are impactful (Chandler and Hopewell 2013) and perceived as being of higher quality and lower bias compared to other types of reviewing genres (Jørgensen et al. 2006). Unfortunately, we know little about citation patterns below the Cochrane level, even if a variety of infrastructures such as reporting guidelines are available, widely used (Page and Moher 2017), and possibly assessable with bibliometric databases (Page et al. 2016). Despite rather small samples, we know little about the citation patterns of review articles being classified as systematic reviews. Are these specific review articles more visible among the scholarly communities, because they are enforced by guidelines and supporting infrastructures?

The infrastructures and organization of review articles, however, consists of more than of guidelines and codes of conduct for the production of tables. A substantial amount of review articles are published in specific types of journals which organize their writing by commissioning them, mostly to highly prestigious authors. Up to now, little is known about the organization and editorial practices surrounding the commissioning of review articles, such as the monitoring of potential authors, the steering of the authors of review articles and the editorial processes in the context of invited submissions. As established above, some authors in the social studies of science have related the practices of commissioning reviews with the reproduction of scientific elite networks (Lievrouw 1988b) termed as invisible colleges (Crane 1972).

Commissioning practices might receive more attention in light of new phenomena related to indicator driven incentive structures, such as the Journal Impact Factor. Again, however, little is known about how the journals react to pressures of increasing or maintaining their position in what has been called “the impact factor game”. Due to their higher impact, a relative increase of the percentage of review publication seems to be one of the most feasible strategies (Falagas and Alexiou 2008; Metze 2010). Since review articles are commonly perceived as highly cited document types (Knottnerus and Knottnerus 2009; Lei and Sun 2020; Miranda and Garcia-Carpintero 2018), there might be temptation to systematically commission review articles to highly reputed authors in order to keep up in that competition. Currently, we find little research about the editorial practices and the relationships between the different groups of a journal which would justify a more substantial claim. Yet, there is indication that the orchestration of document types within journals appears to be a subject of editorial or publisher led governance, as there is increasing research about the management of non-citable items among journals (Moed and van Leeuwen 1995). However, more research on editorial practices and dynamics in document type arrangement are needed in order to systematically account for such phenomena pertaining to the commissioning and organizing of review articles by scholarly journals.

The epistemic role of review articles in author and citation networks

Much research in scientometrics dedicated to review articles has focused on the evaluative effects of review articles, by particularly taking the citation figures and distributions into account. Yet, citation analyses pertaining to reviews have been less often executed for more exploratory purposes, for instance, in order to understand the cognitive structure of a scientific field or a discipline. Yet, review articles may be a promising genre to study such mechanisms. Because review articles are visible representations of their field, it might be promising to use review articles in order to study the canonical structure of a field. Reviews channel citation flows, thus foster certain beliefs within communities and dismiss others (Greenberg 2009). This is not surprising if we consider the role of reviews in the emergence and closure of scientific controversies (Machamer et al. 2000).In which ways do reviews regulate or represent the structure of their discipline?

In addition, workshop results have shown that it might be interesting to study the role of review articles in field formation. Though Bastide et al. (1989) have argued that the existence of review articles may be indicating maturity of scientific fields, the question of when reviews occur might be an interesting one. Different to what de Solla Price (1963) once suggested there may be reasons for their appearances already in early stages of formation. Blümel (2016), for instance, established that reviews may be used to legitimize the establishment of research fields particularly on an early stage, exactly in order to aggregate research streams and to make relationships between different research streams visible. In order to legitimize the claim of novelty and relevance, review articles in novel fields can employ rather visionary and promissory rhetoric. Again, this relates to novel functions review can play not only for informing scholarly communities, but also for reaching out to audiences beyond academia. Finally, drawing from such research, it may even be suitable to use review articles for defining or representing scientific communities in scientometrics. Reviews then might be used to make an a priori definition of a research field, and thus enhance information retrieval.

Review articles have an important function for audiences outside of academia because they are thought to provide an overview over a field and represent scientific consensus. As new data sources beyond scholarly communication, such as Altmetrics, are available, the role of reviews in knowledge translation becomes assessable for informetric analysis (Bornmann and Marx 2015). For example, Bornmann et al. 2016 found, that review articles are an overrepresented document type in policy documents (Bornmann et al. 2016). In medicine, reviews are also important in the education of general practitioners (Hutchison et al. 1995). Further, systematic reviews are thought to provide the most reliable evidence, thus they are increasingly meaningful to policymakers (Fox 2005) and for the creation of clinical guidelines that instruct medical practitioners (Moreira 2007). Although such guidelines become more visible within medical practice and also bibliographic databases (Weisz et al. 2007) the relation between reviews and guidelines is not yet scientometrically assessed. Last but not least, Altmetrics also enables us to study the impact of reviews in nonprofessional communication and already revealed that the document type mix varies between different communication channels (Haustein et al. 2015). However, important questions, such as the actual relation between impact and disciplinary attributions, remain unanswered.

Who writes reviews: authorship patterns of review articles

Finally, we also need more research about who actually writes review articles. There are several hypotheses indicating that review articles are more likely being written by rather established or respected scholars in a given field. But, as the writing of review articles becomes more prestigious and known for being instrumental to gain citation counts, younger scholars may also be inclined to include the writing of review articles in their publication strategy (Miranda and Garcia-Carpintero 2018). Hence, what academic age do authors of review articles have?

Moreover, we know little about the performative effects of the writing of review articles on subsequent specialization in this particular genre of writing. Are authors that have published review articles more inclined to write them again? Do we even find specialists in review writing like Garfield yearned for? In the biomedical field, the rise of the systematic review has led to a larger demand for researchers being able to cope with extending methodological requirements and greater amounts of primary data (Moreira 2007). In addition, the writing of review articles can also result from continued collaboration related to specific authors having a tendency to write those types of texts. Hence, the production of texts of a specific sort may be related to the collaboration with specific author types, thus being perceived as a function of an author’s network embeddedness or relatedness.

Conclusions

Research on review articles has once been characterized being an important resource for scientometrics. Review articles, it was argued may serve several functions beyond the representation of topics and research fields. Yet, within the recent years, research on that genre of communication declined in scientometrics. The review article can be considered as a merely neglected genre in the analysis of scholarly discourse (Azar and Hashim 2014, p. 76; Bastide et al. 1989).

This article has argued that the study of review articles offers nonetheless interesting opportunities to study phenomena of scientific communication and knowledge production. We have reviewed research not only in scientometrics, but also in other fields, in order to detect novel problems, opportunities and future issues for studying review articles in science. Linguistics, sociology, or biomedicine are relevant fields for studying uses and characteristics of review articles. Research in these fields has started to interrogate the uses of reviews in various ways, claiming that these are important, not only for characterizing, but also for communicating research to wider audiences.

Based on these challenges and shortcomings of existing research, we have identified six different fields for future studies of research, methods and methodological strategies for dealing with algorithms in scholarly databases, epistemic strategies for field specific analyses of scientific texts and review articles, full text and content analyses of review articles, studies into the organizational and infrastructural practices of review articles, studies into epistemic dynamics and field formation using review articles, and, finally, studies of review articles authorship. As was shown, there are several indications for dynamics in these streams, since they relate to evolving trends in epistemic organization and scholarly communication. The research agenda on review articles intends to motivate for research on review articles, but we highly encourage to extending research to types of text other than review articles.