Advertisement

Beyond Coverage: Toward a Bibliometrics for the Humanities

  • Björn Hammarfelt
Open Access
Chapter

Abstract

In this chapter, the possibility of using bibliometric measures for evaluating research in the humanities is pondered. A review of recent attempts to develop bibliometric methods for studying the humanities shows that organizational, epistemological differences as well as distinct research practices in research fields ought to be considered. The dependence on colleagues, interdisciplinarity and the ‘rural’ nature of research in many humanistic disciplines are identified as factors that influence the possibilities of applying bibliometric methods. A few particularly promising approaches are highlighted, and the possibility of developing a ‘bibliometrics for the humanities’ is examined. Finally, the intellectual characteristics of specific disciplines should be considered when quality indicators are constructed, and the importance of including scholars from the humanities in the process is stressed.

Keywords

Humanities Bibliometrics Indicators Epistemology Review 

1 Introduction

In this chapter, I argue that bibliometric research on the humanities is now slowly maturing. It appears as if the field is gradually moving from analyzing coverage to a new line of inquiry that tries to understand the humanities on its own terms: looking at specific fields rather than a large heterogeneous collection of disciplines gathered under the label of ‘the humanities’ or ‘the social sciences and the humanities’ (SSH). This new line of research refrains from the familiar, but sometimes unfortunate, distinction between the humanities and the natural sciences, and in doing so abandons the common practices of portraying the social sciences and the humanities as the ‘other’ that does not fit into the bibliometric universe.

The additional focus on the actual characteristics of disciplines has led to attempts to develop bibliometric approaches that are sensitive to the organization of research fields in the humanities. Examples of such attempts include the use of non-source items in established citation databases such as Web of Science (Hammarfelt 2011; Linmans 2010), the use of alternative databases like Google Scholar (Kousha and Thelwall 2009; Koshua et al. 2011) and the recent exploration of the possibilities that the new Book Citation Index offer (Gorraiz et al. 2013; Leydesdorff and Felt 2012). These efforts include exploration of local databases (Engels et al. 2012), references in grant applications (Hammarfelt 2012b), book reviews (Zuccala and van Leeuwen 2011) as well as inclusion in library catalogues (White et al. 2009). Recently, the possibilities that altmetrics offer for the humanities have also been investigated (Hammarfelt 2014; Holmberg and Thelwall 2013; Mohammadi and Thelwall 2013).

The broadening of quality criteria as well as the inclusion of many different types of approaches and materials appear promising. However, this chapter highlights aspects other than methods, materials and coverage as it emphasizes the purpose and organization of research. Thus, I claim that coverage is not the only issue, and maybe not even the most problematic one when discussing the use of bibliometrics on research fields gathered under the heading ‘humanities’.

I begin by outlining the background of bibliometric research on the humanities. I do not claim this overview—which is partly adopted from my dissertation (Hammarfelt 2012a)—to be an extensive review of previous research; instead, I sketch out some of the main findings on the topic. Following this short overview, I discuss recent attempts to develop bibliometric methods that are in tune with research practices in the humanities. These include novel databases, new sources and methods as well as already implemented evaluation systems. In the subsequent section, I introduce theoretical concepts for relating the organization of research fields to publication and citation patterns. Whitleys (2000) theory on the intellectual organization of research as well as Becher and Trowlers (2001) characterization of academic tribes are explicated in this context. I then use these concepts to explain the organization of research in the humanities and its implications for bibliometric measures. Finally, I examine the possibilities of establishing a bibliometrics for the humanities and propose a few suggestions for future research.

1.1 The Humanities

The definition of research fields as either social science or humanities is governed by institutional and epistemological considerations, which further depend on the organization of research in countries or regions. The lists of fields defined as the humanities differ between contexts and countries. The Organization for Economic Co-operation and Development (OECD) lists history, archaeology, genealogy, literature, languages, philosophy, arts, history of arts, religion and theology (OECD 2002, p. 68) while The European Reference Index for the Humanities (ERIH) distinguishes fifteen fields in the humanities (including educational research as well as gender studies and psychology). In the United States, however, the Humanities Resources Center includes eleven fields (Leydesdorff et al. 2011).1

Due to the blurry boundaries of the humanities and the ever-changing disciplinary landscape, no definite collection of fields in the humanities can be given. However, a core of fields—that are on all ‘lists’—can be distilled: art, philosophy, music, language, literary studies and religious studies. These fields are also the ones discussed in this chapter with an additional focus on literary studies. The humanities is a heterogeneous collection of disciplines, and major differences exist between journal-based fields such as linguistics and more book-based fields such as literary studies and religious studies. The conclusions drawn in this chapter concern the latter disciplines rather than more journal-oriented fields such as linguistics and philosophy. I take the liberty of using the term ‘the humanities’ as the topic of enquiry, and this is in line with the majority of previous research on this theme. At the same time, I recognize and discuss the problems that such an approach entails.

2 Bibliometric on the Humanities: A Short Recapitulation

Historically, bibliometric research on the humanities has focused mainly on the inadequate coverage of publications by humanities scholars in available citation databases.2 Several reasons for the scant coverage are mentioned in the literature on the topic: diverse publication channels, the importance of ‘local’ languages as well as the wide-ranging audience of research.

The heterogeneous audience of research is an often-asserted characteristic of scholarship in the humanities. A basic division is often made between publications directed toward fellow researchers and writings directed to a public audience. Nederhof distinguishes the audience further (2006, p. 96) into three groups: international scholars, researchers on the national or regional level and a non-scholarly audience. Another often-cited division is the one suggested by Hicks (2004), in which she separates journal articles, books, national and non-scholarly literature. Her categorization—although originally used to characterize scholarly literature in the social sciences—is also used for describing the humanities. The main difference between these two schemes for describing the varied publications channels and the heterogeneous audience of research is that Nederhof focuses on the ‘target audience’ while Hicks discusses ‘types of literatures’. I propose that focusing on the audience rather than the publication channel allows for a discussion that places the role and purposes of the humanities at the forefront. The three groups suggested by Nederhof also have the advantage of not being clearly separated, as a publication potentially could target all three groups. The categories proposed by Hicks, on the other hand, demand a separation between scholarly and non-scholarly literature. It is also unclear how these groups relate to each other; a book directed to a national and public audience could in theory be categorized as ‘book’, ‘national’ and ‘non-scholarly’ at the same time.

2.1 Publication Patterns

Of special interest in the discussion regarding publication practices in the humanities is the role of the monograph (Lindholm-Romantschuk and Warner 1996; Thompson 2002). The monograph reaches all three audiences to a greater extent than the journal article, and has been deemed especially efficient in targeting non-scholarly readers. Publications directed to a popular audience play an important role, and writing monographs can be seen as an effort to target a scholarly and a popular audience.

However, articles in journals and books are the publication channels most frequently used by researchers in the humanities. Kyviks (2003) study of publication practices among Norwegian scholars in the humanities showed that articles—in books or in periodicals—are the most common output. Articles or chapters in books are also frequent in the social science and the humanities, and a small increase in international (English) and co-authored publications was detected. The recent exploration of publication patterns in the social sciences and humanities in Flanders (Belgium) shows that journal publishing is increasing in the social sciences but decreasing in the humanities. A general increase in the production of publications and especially English language publications was also detected, but no major shift toward publishing in journals was discerned (Engels et al. 2012). Similar results—an increase in the number of international publications (including publications in German or French)—were found in a recent study of publication patterns at the faculty of Arts at Uppsala University in Sweden. Notable from this study was that researchers perceived major changes in publication patterns while the actual changes in publication patterns were small (Hammarfelt and de Rijcke 2015).

2.2 Citing of Sources

A sweeping generalization is that scholars in the humanities mostly publish journal articles and book chapters but cite monographs. Thus, the overlap between citing and cited documents is small in many fields, and it is often reported that scholars in the humanities use older literature as well as primary sources. However, there are notable differences within the humanities in the citing of sources, and the percentage of references to books and edited books varies from 88 % in religion to only 49 % in linguistics (Fig. 1).3
Fig. 1

Percentage of cited books and journal articles in selected fields in the humanities and the social sciences (data from 1995 to 2005). Figure from Hammarfelt (2012a, p. 31)

The earlier findings summarized in the Fig. 1 show that religion, philosophy and literature are book-based disciplines, while journals play an important role in history and linguistics. The overview also shows that books are often cited in social science fields such as sociology and library and information science (LIS). Thus, the problem with counting only citations of journal articles is not restricted to research fields in the humanities.

The extent to which fields in the humanities are adopting referencing practices from the natural sciences has been debated. Larivière et al. (2006) compared the humanities, the social sciences, engineering and the natural sciences in terms of journal publication. The authors found a general increase in journal citations between 1981 and 2000, and this finding applied to the natural sciences and engineering as well as to the social sciences and the humanities. However, when specific fields, such as history, law and literary studies, were examined, a decrease in journal citations during the period was detected.

2.3 The Language and Age of Cited Sources

The language of sources is rarely an issue in the natural sciences since English is the lingua franca. The situation is different in the humanities as many fields in the social sciences and the humanities have a strong regional or national orientation. This is the case especially in fields such as literary studies, sociology and political science (Nederhof 2006 citing Luwel et al. 1999). Databases that predominately index English-language sources cannot adequately cover these fields, and this is a major issue when using established databases such as Web of Science or Scopus to study research fields in the humanities.

Literary studies are a field in which non-English sources play a major role. The influence of English-language sources is moderate: Less than 15 % of the cited sources in German literature and only 9 % of the cited sources in French literature are in English (Cullars 1989). Swedish literary studies has a higher percentage of citations of English-language sources (between 43 and 54 %), but Swedish as well as German and French sources are frequently cited (Hammarfelt 2012b). Consequently, studies of these fields must incorporate non-English sources, and the same applies to many other countries and research fields.

Scholars in the humanities use sources that cover a wide age span. The age of the sources used in research is related to the search for literature, and the pressure to keep up with current research is less pronounced. Thus, a research front is hard to discern, and long time windows are needed when conducting bibliometric analyses. De Solla Price explained the difference in the ‘consumption’ of sources by using a metaphor of digestion: ‘With a low index one has a humanistic type of metabolism in which the scholar has to digest all that has gone before, let it mature gently in the cellar of wisdom, and then distill forth new words of wisdom about the same sort of questions’ (de Solla Price 1970, p. 15). This characterization disregards the diversity of research in the humanities, although the metaphor of digestion is illustrative. Furthermore, Price overlooked that many sources in the humanities are primary sources (for example, historical sources and literary works), which increases the median age of the sources considerably.

Bibliometric studies of the humanities show that the type of publication most frequently cited is the monograph, the age span of the cited references is broad and languages other than English play a significant role in many fields (Hammarfelt 2012a). These characteristics are agreed upon by many, but several matters remain unresolved. One question is whether the publication practices of scholars in the humanities are adapting to the norms that prevail in the natural sciences. A few studies (Butler 2003; Kyvik 2003) suggest that this might be the case, while others emphasize the constancy of cited and published material (Hammarfelt and de Rijcke 2015; Larivière et al. 2006). How the increasing importance of ‘research outputs’ across research fields will influence publication practices in the humanities has not been determined. However, implementing publication-based performance measures will undoubtedly put further focus on this issue, and perhaps this will lead to in-depth studies of the effect that evaluation systems have on scholarship in the humanities.

3 In Pursuit of a Bibliometric for the Humanities

In this section, I briefly present several recent attempts to apply bibliometric methods to the humanities. In addition to being current, the selected studies also have a further sensitivity to the characteristics of research in the humanities in common. Thus, these studies are not only examples of bibliometrics applied to the humanities but also to some extent examples of bibliometric methods developed ‘for the humanities’. A general feature of these attempts are an effort to introduce new sources for bibliometric analysis, sources that go beyond journals indexed in citation databases such as Thomson Reuters’ Web of Science or Elsevier’s Scopus.

3.1 Book Citation Index

An obvious solution to the problem of low coverage of non-journal publications in citation indexes is to start indexing books. The launch of the Book Citation Index in 2011 is an attempt to improve the coverage of the humanities, and it could open up for analysis of how the journal literature and the book literature relate to each other. However, the index still has a very limited scope, mainly English-language sources are included (Gorraiz et al. 2013), and problems remain when distinguishing between different types of books. Initial studies have also found that the citation rates of books are low in many research fields (Leydesdorff and Felt 2012). Thus, the current Book Citation Index is of little use for evaluating research but might provide valuable knowledge regarding the relation between journal literature and books.

3.2 Non-source Items

It was possible to track citations of books that are not indexed in citation databases, before the launch of the Book Citation Index. Citation of so-called ‘non-source’ items has been used for studying impact and interdisciplinarity (Hammarfelt 2011; Linmans 2010). However, this method involves limitations on the size of the material used, and considerable data cleaning is needed, since the cited sources are not standardized. Another constraint of this method is that it gathers citations only from a small portion of the literature in many research fields in the humanities. The approach is in principle restricted to English-language publications, and the analysis of ‘non-source’ items is limited to small data sets due to the manual work involved.

3.3 Google Scholar, Google Book Search

An alternative to the use of traditional citation indexes is options such as Google Scholar (GS) or Google Book Search (Kousha and Thelwall 2009; Koshua et al. 2011). The main constraints of GS are that analyses cannot be automatized and the data is hard to process. Every post has to be checked, and new searches for each publication are required. The benefit of Google Scholar is greater coverage—which includes books—and that everyone is free to use the database (with limitations on what you can do). The reliability of the data is a concern since inflated citation counts as well as ghost authors and ‘phantom authors’ limit the usability of the data for bibliometric analysis (Jacso 2010).

3.4 Ad Hoc Databases

A response to the limits of existing data sources is to build your own citation database. When targeting specific contexts—Catalan literature (Ardanuy et al. 2009) or Swedish literary studies (Hammarfelt 2012b)—this method might be viable. The building of ‘ad hoc databases’ allows analyses of materials that usually are not indexed in citation indices such as grant applications (Hammarfelt 2012b), and small local studies can provide valuable contrast to larger studies of citation patterns. However, the amount of labor involved in harvesting references by hand and then indexing them in a database inherently limits the size of the datasets used.

3.5 Library Catalogues

Several authors have suggested that library catalogues might be a possible data source for evaluating the impact of books (Linmans 2010; Torres-Salinas and Moed 2009; White et al. 2009). The basic idea is simple: The more libraries that stock a book, the more influential it is deemed to be. The inclusion of a book in a catalogue indicates that the book is judged important. However, implementing the model on a larger scale would be difficult: Libraries do not always make informed judgments when buying books; they often buy bundles of books. The model does not include open access or e-books, and an evaluation system based on this approach would put the librarians making the buying decisions in a delicate position. Furthermore, one could imagine that authors and publishers could easily manipulate such a system.

3.6 Book Reviews

Book reviews have an important gatekeeping function in the humanities, and reviews are often seen as an important merit and indicator of influence for the author writing the review. Book reviews have also been proposed as an important unit of analysis when it comes to book-oriented fields. Zuccala and van Leeuwen (2011) proposed that the number of book reviews produced by a researcher can be seen as a measure of success. One problem though is that already established and older researchers often are those invited to review books. Thus, a system that counts written reviews could disadvantage younger and less renowned scholars. Another alternative is to view book reviews as ‘mega-citations’ that indicate the quality of a book (Zuccala et al. 2014). This approach has many advantages, especially since book reviews play an important function in the humanities; however, many books are never reviewed, and the overall coverage is possibly too low for systematic assessment.

3.7 Counting and Weighing Publications

An alternative of course is to not use citations at all and instead count publications. This system makes it possible to evaluate research in all fields independently of publication channel and language. A qualitative aspect can be introduced in order to circumvent the flourishing of low-quality publications. The idea of weighing publication according to type and channel has been proposed by Finkenstaedt (1990) and Moed et al. (2002). However, the most well-known and influential system for counting and weighing publications is the Norwegian one (Schneider 2009; Sivertsen 2010). This system is used for allocating resources among universities in Norway. The main benefits of the system are the coverage of publications, transparency and the adaptability of the system (Ahlgren et al. 2012). However, many publications in the humanities are still not included due to the definition of ‘scholarly literature’, and monographs at prestigious ‘non-academic’ publishers are seldom counted. The consequence is that a lower share of the total publications by humanities scholars is covered by the system. This disadvantage is partly compensated by publications being fractionalized over authors, which has shown to benefit scholars in the humanities compared to disciplines where co-authorship is common (Piro et al. 2013).

3.8 Altmetric Approaches

Altmetrics—metrics based on data from the social web—is a promising approach in the efforts to find appropriate methods for assessing the humanities (Tang et al. 2012). These new, ‘altmetric’ measures propose not only to solve problems with established methods but also to measure impact beyond citations from academic journals. One of the most popular data sources used for altmetric analysis is Twitter. Holmberg and Thelwall (2013) found that scholars in the history of science were less likely to use Twitter for scholarly purposes compared with other fields, and across all fields, few tweets contained links or mentions of scholarly literature. Another common source of altmetric data is the social reference manager Mendeley, but the coverage for humanities articles was also quite low (28 %) when compared to the social sciences (58 %) (Mohammadi and Thelwall 2013). The inclusion of many different types of sources, the ability to study impact beyond the scholarly realm, as well as the openness of many services appear promising for the humanities. However, limitations remain with the dominance of English-language journal articles the most significant (Hammarfelt 2014).

There is no shortage of approaches for studying the humanities with bibliometric methods, and the brief orientation given here is not exhaustive. Still, the overview illustrates that bibliometric research depends on the availability of data sources, especially citation indices, and the content, availability and coverage of these data sources dictate how research is conducted. Thus, many of the studies mentioned were influence by the introduction of new services such as Google Book Search, Google Scholar, or The Book Citation Index. The research field of bibliometrics can be duly criticized for its dependence and focus on available data sources, even more as these services are provided by private companies and, thus, are not easily adapted to the fields needs by scholars themselves. However, the main purpose of bibliometric research is not to study databases or coverage, but to further our understanding of communication structures in science and research. In this effort, we have to go beyond issues of database content and coverage and focus on the organization and characteristics of research in different disciplines. Accordingly, in the following chapter I reflect on publication patterns and referencing practices in relation to the social and intellectual organization of research fields.

4 Intellectual Organization of Research Fields and Its Bibliometric Consequences

In the following section, I describe how publication practices and citation patterns can be understood from a disciplinary perspective where the use of references depends on how a research field is organized. The characterization of research fields in the humanities suggested by Whitley (2000) and Becher and Trowler (2001) is briefly reviewed, and related to publication patterns and referencing practices. However, the vast difference between research fields and subfields gathered under the umbrella of the humanities should be acknowledged, and the generalizations made here apply foremost to literary studies and similar book-based disciplines.

4.1 Fragmented and Rural Research Fields

The majority of disciplines within the humanities are in Whitleys characterization defined as fragmented adhocracies. These fields are intellectually varied as well as heterogenic since research in fragmented adhocracies is personal and poorly coordinated, and the degree of specialization is limited. The dominant attribute of these fields is the lack of a stable configuration; tasks are not specialized; co-ordination is weak, and when it occurs, it is based on personal relations (Whitley 2000). Subgroups form around specific topics and discrete methodological approaches. Audiences are varied, and so are the methods used. There is considerable disagreement on which topics to study as well as on how these topics should be approached, and the lack of standards makes it difficult to resolve disputes.

Another useful characterization for understanding the organization of research fields is the one between rural and urban fields (Becher and Trowler 2001). The distinction between rural and urban concerns the ‘density’ of a discipline or a research area; if many researchers are working on the same problem, then the research area is described as urban, while a less populated discipline is deemed rural. Strong competition for positions and resources can be observed in an urban research area (for example, biomedicine), whereas there are fewer struggles for resources and recognition (as well as fewer rewards) in rural fields.

4.2 Referencing Practices and Citation Patterns

I propose that referencing practices and citation patterns are further understood by the intellectual characteristics of the research field: A less demarcated discipline lacking a central core is heavily influenced by other research fields and therefore more interdisciplinary in referencing practices. Citation patterns are also determined by the number of researchers engaged on a specific topic: In an urban field, it is important to keep up with the ‘research front’ and cite recent literature, while the age of sources plays less of a role in rural fields. This is also connected to the speed of publication, which is considerably faster in an urban field (biomedicine) than in a rural one (literary studies) (Table 1).
Table 1

Characteristics of the humanities and influence on publication and citation patterns

Field characteristics

Publication patterns

Referencing practices

Low dependence on colleagues

Various publication channels; importance of public audience

Interdisciplinary references common

Rural organization

The pace of publications is slow

Citations gather slowly; number of ‘possible citations’ is low

Another variable that influences referencing practices is the audience. In fields where a non-academic audience plays an important role, scholars may choose a referencing style—the footnote is an example—that serves a scholarly and a popular audience. The degree of dependence between researchers and the definition of originality also affect the use of references. It is important to cite colleagues in a field where researchers depend on each other for recognition and rewards, but in fields where originality is highly valued, referencing serves other purposes as well (Hellqvist 2010).

Thus, two main characteristics that influence referencing practice and citation patterns in the humanities can be discerned: low dependence on colleagues and the rural organization of the field. The varied audience, rural organization and low dependence on colleagues are related. A diverse audience makes it possible for individual researchers to find readers outside their own field, with the consequence that scholars depend less on peers for recognition. The high task uncertainty of many fields in the humanities and the low dependence on colleagues give the individual scholar great freedom in pursuing a unique research profile, which results in researchers being scattered across many different topics with little communication between them. Thus, scholars in the humanities enjoy many possibilities when selecting topics, publication channels and whom to cite, but this in turn limits the potential of receiving ‘rewards’ in the form of citations. The low coverage of publications in citation databases is therefore not the most important reason why citation scores are less applicable as an indicator of impact in the humanities. Instead, I propose that the social and intellectual organization of the humanities is the main reason to why citation-based approaches are less applicable in these fields.

5 Conclusions

The bibliometric community has rightly discouraged the use of conventional bibliometric methods for evaluating the humanities. Especially, citation analysis using journals indexed in citation databases is less applicable in these fields. This conclusion is firmly based on several studies showing that the coverage of the humanities in databases such as Web of Science or Scopus is insufficient for evaluation and not representative of research in the humanities. Research assessment systems, such as the one used in Norway, amend this by including all scholarly publications. The publications are then given points depending on the publication channel (monograph, anthology, or journal) and the ‘quality level’ of the journal or the publisher. However, the definition of what should count as a ‘scholarly publication’ is still a matter of debate. There is no consensus on what an important research output is in the humanities; a peer-reviewed journal article in an international journal, a book chapter in an anthology edited by a renowned scholar, or a monograph at a prestigious non-academic publisher can all be seen as important outputs, and publications directed toward a popular audience are often highly rated. Consequently, the choice of publications that should be valued in assessing research depends on our view of the humanities and its overall purpose in society.

A recurrent problem in evaluating the humanities is the long time span needed for measuring the impact of research. The lifetime, as well as the distribution of citations to a publication over time, must be considered. Research by humanities scholars may be used in twenty, fifty, or even a hundred years, but sustainability is seldom measured in research assessment exercises. Thus, a considerable part of research in the humanities—such as the preservation and translation of cultural heritage—might be valuable for future generations, but it is invisible in the limited perspective of research evaluation.

The development of bibliometric methods that fairly capture the ‘impact’ of research involves understanding how research is organized in these fields. This is confirmed by the findings recapitulated that point to differences in intellectual organization, and in the actual use of references as major reasons for why citation-based approaches are less applicable to the humanities. Thus, in developing bibliometric methods that accurately depict the humanities, we must go beyond the issue of coverage and focus on the social and intellectual organization of the fields involved. However, there are vast differences in research practices within the humanities, and differences are also evident among specialties within the same discipline. Furthermore, research practices are constantly changing due to technical developments (digitalization), external demands (research evaluation, open access) and internal negotiations on the purpose of research. Research on scholarly communication—including bibliometric approaches—is needed in order to follow these developments. Furthermore, when studying scholarly practices, we must be careful not to be caught in old dichotomies that portray ‘two cultures’, but acknowledge that research across all disciplines shares many similarities. The need for fair and reliable assessment methods cuts across all research fields, and constructing indicators that properly capture the quality and impact of research is challenging for academia at large.

Constructing appropriate indicators involves actively engaging the researchers being evaluated. Recent attempts at identifying quality indicators in the humanities show that the ‘notion’ of quality is not easily captured, and several conflicting norms were found (Ochsner et al. 2013). The construction of general and all-encompassing indicators is hindered by the heterogeneous nature of research as well as differences in how quality is perceived. However, alternatives to the use of peer review, which not only is time-consuming but also prone to reinforce established hierarchies, are needed in the humanities. Here I believe evaluations that use bibliometrics might provide a valuable complement to traditional peer review, but only if the indicators used are carefully constructed in a dialog with the researchers being evaluated.

5.1 Challenges

Bibliometrics may play an important role in future attempts to study the wider impact of research in the humanities, and citation analysis could be used to further our understanding of the organization and development of research in these fields. Approaches such as using citations to ‘non-source items’, introducing new databases and services, and using altmetric measures all appear promising but are far from utilizable on a general level. These and several other innovative techniques for studying the humanities have been identified in this chapter, and one argument made is that bibliometric research on the humanities has become more attuned to the scholarly tradition of humanistic scholarship. Still, much must be done to study and assess the humanities, and I identify a few areas that are particularly interesting for future research.

First, I suggest that it is time to devote attention to more detailed and restricted areas of research. It is less complicated to define fields and delineate ‘subfields’ in the natural sciences, and this might be one reason for using a broad and inclusive definition when studying the humanities. Extensive interdisciplinary citing might be another reason for adopting ‘the humanities’ as the object of study. However, I propose that focusing further on specific fields and specialties would yield a better understanding of publication and citation patterns in the humanities. I also envision that developing new and more accessible bibliometric tools and approaches will result in further application of bibliometric methods by humanist scholars themselves.

Altmetric methods that are in tune with the organization of the humanities is an additional area for research. Attempts at actually systematically measuring social impact—impact outside academia—are promising. Such measures would be an important contribution not only for assessing the humanities but also for measuring the general influence of research in society. Exploring sources, mainly books and non-English language publications that are seldom covered by traditional bibliometric approaches is another exciting vein of research. Altmetrics is a very novel phenomenon and its ability to measure quality or impact is still debated, but the general ambition of including many different types of sources that measure impact in a multifold of ways is encouraging for the efforts to develop ‘metrics’ for the humanities.

Finally, the meeting of a ‘metric culture’ with scholarship in the humanities is a particularly important area of study. For a long time, the natural sciences have lived with impact factors, and researchers in these fields often calculate their own H-index. However, scholars in the humanities are less familiar with bibliometric measures, and many researchers not only fear unfair rankings and evaluations but also often see them as alien to humanistic scholarship. Thus, a crucial topic is how the organization and character of the humanities will respond to additional measurement and assessment attempts. The answer to this question is important not only for the bibliometric community but also for the future of scholarship in the humanities.

Footnotes

  1. 1.

    These fields are English language and literature, foreign languages and literature, history, philosophy, religion, ethnic-, gender- and cultural studies, American studies and area studies, archeology, jurisprudence, selected arts and selected interdisciplinary studies.

  2. 2.

    For an orientation in the wider literature on the evaluation of the humanities, the reader can consult the Arts and Humanities Research Assessment Bibliography (Peric et al. 2013), which currently has a little over a thousand publications indexed, Nederhof (2006) provides a review of issues regarding bibliometric evaluation, and recently a bibliography of research on the humanities and bibliometrics covering the years 1940–2010 was provided by Ardanuy (2013).

  3. 3.

    Data collected from several previous studies: religion (Knievel and Kellsey 2005), philosophy (Cullars 1998), music (Knievel and Kellsey 2005), literature (Thompson 2002), arts (Knievel and Kellsey 2005), history (Lowe 2003), sociology (Lindholm-Romantschuk and Warner 1996), LIS (Chung 1995) and linguistics (Georgas and Cullars 2005).

Notes

Acknowledgments

This chapter builds on findings from my dissertation, Following the Footnotes: A bibliometric analysis of citation patterns in literary studies (2012), and segments of the text are redrafted and shortened versions of arguments found there.

References

  1. Ahlgren, P., Colliander, C., & Persson, O. (2012). Field normalized rates, field normalized journal impact and Norwegian weights for allocation of university research funds. Scientometrics, 92(3), 767–780. doi: 10.1007/s11192-012-0632-x.
  2. Ardanuy, J. (2013). Sixty years of citation analysis studies in the humanities (1995–2010). Journal of the American Society for Information Science and Technology, 64(8), 1751–1755. doi: 10.1002/asi.22835.CrossRefGoogle Scholar
  3. Ardanuy, J., Urbano, C., & Quintana, L. (2009). Citation analysis of Catalan literary studies (1974–2003): Towards a bibliometrics of humanities studies in minority languages. Journal of the American Society for Information Science and Technology, 81, 347–366. doi: 10.1007/s11192-008-2143-3.Google Scholar
  4. Becher, T., & Trowler, P. R. (2001). Academics tribes and territories: Intellectual enquiry and the cultures of disciplines. Buckingham: Open University Press.Google Scholar
  5. Butler, L. (2003). Explaining Australia’s increases share of of ISI publications—the effects of a funding based on publication counts. Research Policy, 32, 143–155. doi: 10.1016/S0048-7333(02)00007-0.CrossRefGoogle Scholar
  6. Chung, Y.-K. (1995). Characteristics of references in international classification systems literature. The Library Quarterly, 65(2), 200–215.CrossRefGoogle Scholar
  7. Cullars, J. M. (1989). Citation characteristics of French and German literary monographs. Library Quarterly, 59 (305–325).Google Scholar
  8. Cullars, J. M. (1998). Citation characteristics of English-language monographs in philosophy. Library and Information Science Research, 20(1), 41–68. doi: 10.1016/S0740-8188(98)90005-6.CrossRefGoogle Scholar
  9. de Solla Price, D. J. (1970). Citation measures of hard science, soft science, technology, and nonscience. In C. E. Nelson & D. K. Pollock (Eds.), Communication among scientists and engineers (pp. 3–22). Lexington, MA: Heath.Google Scholar
  10. Engels, T. C., Ossenblok, T. L., & Spruyt, E. H. (2012). Changing publication patterns in the social sciences and humanities, 2000–2009. Scientometrics, 93(2), 373–390. doi: 10.1007/s11192-012-0680-2.CrossRefGoogle Scholar
  11. Finkenstaedt, T. (1990). Measuring research performance in the humanities. Scientometrics, 19(5–6), 409–417. doi: 10.1007/BF02020703.CrossRefGoogle Scholar
  12. Georgas, H., & Cullars, J. M. (2005). A citation study of the characteristics of the linguistics literature. College and Research Libraries, 66(6), 496–515. doi: 10.5860/crl.66.6.496.CrossRefGoogle Scholar
  13. Gorraiz, J., Purnell, P. J., & Glänzel, W. (2013). Opportunities for and limitations of the book citation index. Journal of the American Society for Information Science and Technology, 64(7), 1388–1398. doi: 10.1002/asi.22875.CrossRefGoogle Scholar
  14. Hammarfelt, B. (2011). Interdisciplinary and the intellectual base of literature studies: Citation analysis of highly cited monographs. Scientometrics, 86(3), 705–725. doi: 10.1007/s11192-010-0314-5.CrossRefGoogle Scholar
  15. Hammarfelt, B. (2012a). Following the footnotes: a bibliometric analysis of citation patterns in literary studies. (Doctoral dissertation. Skrifter utgivna vid institutionen för ABM vid Uppsala Universitet, Vol. 5). Uppsala: Uppsala Universitet. Retrieved from http://www.diva-portal.org/smash/get/diva2:511996/FULLTEXT01.pdf.
  16. Hammarfelt, B. (2012b). Harvesting footnotes in a rural field: citation patterns in Swedish literary studies. Journal of Documentation, 68(4), 536–558. doi: 10.1108/00220411211239101.CrossRefGoogle Scholar
  17. Hammarfelt, B. (2014). Using altmetrics for assessing research impact in the humanities. Scientometrics, 101(2), 1419–1430. doi: 10.1007/s11192-014-1261-3.
  18. Hammarfelt, B., & de Rijcke, S. (2015). Accountability in context: Effects of research evaluation systems on publication practices, disciplinary norms, and individual working routines in the faculty of Arts at Uppsala University. Research Evaluation, 24(1), 63–77. doi: 10.1093/reseval/rvu029.CrossRefGoogle Scholar
  19. Hellqvist, B. (2010). Referencing in the humanities and its implications for citation analysis. Journal of the American Society for Information Science and Technology, 61(2), 310–318. doi: 10.1002/asi.21256.Google Scholar
  20. Hicks, D. (2004). The four literatures of social science. In H. F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research: The use of publication and patent statistics in studies of S&T systems (pp. 476–496). Dordrecht: Kluwer Academic Publishers.Google Scholar
  21. Holmberg, K., & Thelwall, M. (2013). Disciplinary differences in Twitter scholarly communication. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. F. Moed (Eds.), Proceedings of ISSI 2013 Vienna: 14th International Society of Scientometrics and Informetrics Conference, Vol. 1, pp. 567–582. Vienna: Austrian Institute of Technology GmbH.Google Scholar
  22. Jacso, P. (2010). Metadata mega mess in Google Scholar. Online Information Review, 34(1), 175–191. doi: 10.1108/14684521011024191.CrossRefGoogle Scholar
  23. Knievel, J. E., & Kellsey, C. (2005). Citation analysis for collection development: A comparative study of eight humanities fields. The Library Quarterly, 75(2), 142–168.CrossRefGoogle Scholar
  24. Koshua, K., Thelwall, M., & Rezaie, S. (2011). Assessing the citation impact of books: The role of Google Books, Google Scholar and Scopus. Journal of the American Society for Information Science and Technology, 62(11), 2147–2164. doi: 10.1002/asi.21608.CrossRefGoogle Scholar
  25. Kousha, K., & Thelwall, M. (2009). Google book search: Citation analysis for social science and the humanities. Journal of the American Society for Information Science and Technology, 60(8), 1537–1549. doi: 10.1002/asi.21085.CrossRefGoogle Scholar
  26. Kyvik, S. (2003). Changing trends in publishing behavior among university faculty, 1980–2000. Scientometrics, 58(1), 35–48. doi: 10.1023/A:1025475423482.CrossRefGoogle Scholar
  27. Larivière, L., Archambault, È., Gringas, Y., & Vignola-Gagné, È. (2006). The place of serials in referencing practices: Comparing the natural sciences and engineering with social sciences and humanities. Journal of the American Society for Information Science and Technology, 57(8), 997–1004. doi: 10.1002/asi.20349.CrossRefGoogle Scholar
  28. Leydesdorff, L., & Felt, U. (2012). ’Books’ and ’book chapters’ in the book citation index (BKCI) and science citation index (SCI, SoSCI, A&HCI). In Proceedings of the ASIS&T Annual Meeting, Vol. 49. doi: 10.1002/meet.14504901027.
  29. Leydesdorff, L., Hammarfelt, B., & Salah, A. A. A. (2011). The structure of arts and humanities citation index: A mapping on the basis of aggregated citations among 1157 journals. Journal of the American Society for Information Science and Technology, 62(12), 2414–2426. doi: 10.1002/asi.21636.CrossRefGoogle Scholar
  30. Lindholm-Romantschuk, Y., & Warner, J. (1996). The role of monographs in scholarly communication: An empirical study of philosophy, sociology and economics. Journal of Documentation, 52(4), 389–404. doi: 10.1108/eb026972.CrossRefGoogle Scholar
  31. Linmans, J. A. M. (2010). Why with bibliometrics the humanities does not need to be the weakest link. Indicators for research evaluation based on citations, libarary holdings and productivity measures. Scientometrics, 83(2), 337–354. doi: 10.1007/s11192-009-0088-9.CrossRefGoogle Scholar
  32. Lowe, S. M. (2003). Reference analysis of the American Historical Review. Collection Building, 22(1), 13–20. doi: 10.1108/01604950310457168.CrossRefGoogle Scholar
  33. Luwel, M., Moed, H. F., Nederhof, A. J., De Samblanx, V., Verbrugghen, K., & van der Wurff, L. J. (1999). Towards indicators of research performance in the social sciences and humanities. An exploratory study in the fields of law and linguistics at Flemish universities. Leiden: Centre for Science and Technology Studies.Google Scholar
  34. Moed, H. F., Luwel, M., & Nederhof, A. J. (2002). Towards research performance in the humanities. Library Trends, 50(3), 498–520.Google Scholar
  35. Mohammadi, E., & Thelwall, M. (2013). Assessing the Mendeley readership of social sciences and the humanities. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. F. Moed (Eds.), Proceedings of ISSI 2013 Vienna: 14th International Society of Scientometrics and Informetrics Conference, Vol. 1, pp. 200–214. Vienna: Austrian Institute of Technology GmbH.Google Scholar
  36. Nederhof, A. J. (2006). Bibliometric monitoring of research performance in the social sciences and the humanities: A review. Scientometrics, 66(1), 81–100. doi: 10.1007/s11192-006-0007-2.CrossRefGoogle Scholar
  37. Ochsner, M., Hug, S. E., & Daniel, H.-D. (2013). Four types of research in the humanities: Setting the stage for research quality criteria in the humanities. Research Evaluation, 22(4), 79–92. doi: 10.1093/reseval/rvs039.Google Scholar
  38. OECD. (2002). Frascati manual: Proposed standard practice for surveys on research and experimental development. Paris: OECD.Google Scholar
  39. Peric, B., Ochsner, M., Hug, S. E., & Daniel, H.-D. (2013). Arts and Humanities Research Assessment Bibliography (AHRABi). Zürich: ETH Zurich. doi: 10.3929/ethz-a-010610785.
  40. Piro, F. N., Aksnes, D. W., & Rorstad, K. (2013). A macro analysis of productivity differences across fields: Challenges in the measurement of scientific publishing. Journal of the American Society for Information Science and Technology, 64(2), 307–320. doi: 10.1002/asi.22746.
  41. Schneider, J. W. (2009). An outline of the bibliometric indicator used for performance-based funding of research institutions in Norway. European Political Science, 8(3), 364–378. doi: 10.1057/eps.2009.19.CrossRefGoogle Scholar
  42. Sivertsen, G. (2010). A performance indicator based on complete data for scientific publication output at research institutions. ISSI Newsletter, 6(1), 22–28.Google Scholar
  43. Tang, M., Wang, C., Chen, K., & Hsiang, J. (2012). Exploring alternative cyber bibliometrics for evaluation of scholarly performance in the social sciences and humanities in Taiwan. In: Proceedings of the ASIS&T Annual Meeting, Vol. 49. doi: 10.1002/meet.14504901060.
  44. Thompson, J. W. (2002). The death of the scholarly monograph in the humanities? Citation patterns in literary scholarship. Libri, 52(3), 121–136. doi: 10.1515/LIBR.2002.121.CrossRefGoogle Scholar
  45. Torres-Salinas, D., & Moed, H. F. (2009). Library catalogue analysis as a tool in studies of social sciences and humanities: An exploratory study of published book titles in economics. Journal of Informetrics, 3(1), 9–26. doi: 10.1016/j.joi.2008.10.002.CrossRefGoogle Scholar
  46. White, H. D., Boell, S. K., Yu, H., Davis, M., Wilson, C. S., & Cole, F. T. H. (2009). Libcitations: a measure for comparative assessment of book publications in the humanities and social sciences. Journal of the American Society for Information Science and Technology, 60(6), 1083–1096. doi: 10.1002/asi.21045.CrossRefGoogle Scholar
  47. Whitley, R. (2000). The intellectual and social organization of the sciences (2nd ed.). Oxford: Oxford University Press.Google Scholar
  48. Zuccala, A., & van Leeuwen, T. (2011). Book reviews in humanities research evaluations. Journal of the American Society for Information Science and Technology, 62(10), 1979–1991. doi: 10.1002/asi.21588.CrossRefGoogle Scholar
  49. Zuccala, A., van Someren, M., & van Bellen, M. (2014). A machine-learning approach to coding book reviews as quality indicators: Toward a theory of megacitation. Journal of the American Society for Information Science and Technology, 65(11), 1643–2330. doi: 10.1002/asi.23104.Google Scholar

Copyright information

© The Author(s) 2016

Open Access This chapter is distributed under the terms of the Creative Commons Attribution-Noncommercial 2.5 License (http://creativecommons.org/licenses/by-nc/2.5/) which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

The images or other third party material in this chapter are included in the work’s Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work’s Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.

Authors and Affiliations

  1. 1.University of BoråsBoråsSweden

Personalised recommendations