Keywords

Bibliometrics is loved neither in the natural sciences, nor in the life sciences, nor in engineering. However, it is a more or less common practice in all of these areas of research. In the humanities and some social sciences, it is neither loved nor practiced—to put it simply. The situation hasn’t changed since the European Research Index in the Humanities’ (ERIH)Footnote 1 was established in 2002. ERIH was established both for humanities ‘purposes and in order to present their ongoing research achievements systematically to the rest of the world’. The Index adds: ‘It is also a unique project because, in the context of a world dominated by publications in English, it highlights the vast range of world-class research published by humanities researchers in the European languages’. It was, and is, its major goal to improve the unsatisfactory coverage of European Humanities’ research through better bibliometric tools.

In 2009, Bonnie Wheeler, President of the Council of Editors of Learned Journals, raised serious objections against ERIH (Zey 2010). She argued: ‘ERIH claims that its goal is to aid journals and their contributors, but it will inevitably inform institutional assessments and may result in rigid common protocols for scholarly journals’ (Wheeler 2009; cf. Wheeler 2011). Wheeler’s concerns are those of many editors regardless of whether their journals are ranked in the ERIH list or not. Maybe not the best, but certainly the most common argument is a different one: In principle, research output in the humanities is not countable and even social sciences are to be treated differently from the science, technology, engineering and medicine (STEM) disciplines. Finally, there is an incongruity between the steadily growing numbers of publications and the need for a fair and effective practice of peer review for sufficient library budgets and preservation services. Because the entire system is heavily dependent on tax-payer money, research organizations are calling for an alternative. They advocate for university-based and open-access publishing models (Harley and Krzys Acord 2011). Not only bibliometrics, but the whole system of scholarly publication is challenged and will be under much more pressure in the next few years than it is today (Leydesdorff 2001).

The Agence National de la Recherche (ANR), the Arts and Humanities Research Council (AHRC), the Deutsche Forschungsgemeinschaft (DFG), the Economic and Social Research Council (ESRC) and the Nederlandse Organisatie voor Wetenschappelijk Onderzoek (NOW) are working together with the European Research Foundation to meet the challenges presented by the current pressure to establish a more robust bibliometric database for assessing the impact of all types of research output in the domains of social sciences and humanities (SSH). They ask how a bibliometric database for the humanities and social sciences can be developed that more accurately represents humanist work than current citation indices like ERIH or newer ‘usage’ indices. A European scoping project was established in 2009 to answer the question: ‘What is the potential for developing some form of research output database that could be used for assessing research performance in SSH?’ In the field of social sciences and humanities the main problems are well known, i.e. the wider scale and variety of research outputs from SSH, the need to consider national journals (in particular those published in languages other than English) and the highly variable quality of existing SSH bibliographical databases due to the lack of a standardized database structure for the input data. On the other hand, it’s obvious how rapidly Web of Science (Thomson-Reuters), which is the former Science Citation Index/Social Sciences Citation Index/Arts and Humanities Citation Index, and Scopus (Elsevier) have expanded their coverage of social sciences and humanities journals in the last years. Web of Science has increased the covered number of SSH journals from 1,700 in 2002 to 2,400 in 2009. And Scopus, much stronger in the field, added 1,450 SSH journals in 2009 to its collection of more than 3,500 SSH journals. Moreover, Scopus has already started to add bibliographic meta-data on highly cited books in its database. So-called regional journals are an increasing part of these two main bibliometric database providers. In March 2014, Elsevier indexed 30,000 books, expecting to index around 75,000 by the end of 2015 (Scopus blog, see Dyas 2014). And, as Henk Moed puts it, Google is already the poor man’s bibliometrics (Moed et al. 2010, p. 19; cf. Harzing and van der Wal 2009). The driving force, however, is the interest of many researchers and universities to make their results more visible.

Within this situation, the European Scoping Project (cf. SPRU 2009) understands bibliometrics in a broad sense, from bibliographic to statistics, and has taken political, strategic and operational issues into account. Two experts—Diana Hicks and Henk Moed—were asked to give a short report on the actual situation of SSH bibliometrics (Hicks and Wang 2009; Moed et al. 2010). After having discussed the evaluations by Hicks and Moed, the scoping project board members developed a variety of solutions and examined more closely six suggestions: First, to create more comprehensive national bibliographic systems through the development of institutional repositories. Second, to enhance and build upon existing national documentation systems like METIS in the Netherlands or the DRIVER initiative through the creation and standardization of institutional research management systems. The third suggestion discussed the possibilities for a new database of SSH research outputs from publishers’ archives and institutional repositories, and adding to this appropriate data on enlightenment literature and curated events. A further point considered was to take advantage of the competition between Web of Science and Scopus to strengthen the coverage of SSH research outputs, and of the potential of Google Scholar to become a more rigorous bibliometric database provider. The fifth suggestion was whether it would be suitable to integrate the specialized SSH bibliographic lists into one comprehensive bibliographic database. And last, there was a discussion on the chances to encourage the further development of the Open Access approach, since it offers a potential means to overcome barriers of accessibility and to enhance the visibility of SSH journals and books published by small European publishers.

Advantages and disadvantages of each approach were weighed and recommendations were given. These recommendation were based on a combination of top-down and bottom-up actions, with an emphasis on extensive bottom-up involvement in the development of an SSH bibliometric database. Main functions of the recommendations were to provide accountability with regard to the use of public funds, to assess research quality, to provide a comprehensive overview of SSH research outputs in Europe, to map the directions of SSH research and to identify new emerging areas of interdisciplinary SSH research. The four recommendations were:

  1. 1.

    Defining the criteria for inclusion of SSH research outputs and establishing a standardized database structure for national bibliometric databases;

  2. 2.

    exploring the option of involving a commercial supplier in the construction of a single international SSH bibliometric database;

  3. 3.

    conducting a pilot study of one or several specific SSH disciplines; and

  4. 4.

    longer-term expansion and enhancement of the SSH bibliometric database.

The required actions for each recommendation were laid out, to mark very concrete further steps. The roadmap was described as a two year path towards a bibliometric database for the humanities and social sciences. The full report was published with both research reports by Moed and Hicks (Martin et al. 2010; Moed et al. 2010; Hicks and Wang 2009).

The European Science Foundation has already reacted and recently signed a memorandum of understanding with the Norwegian Social Science Data Services (NSD). The decision was made to transfer the ERIH to the NSD website, where it will be possible to submit new journals. However, no decision has been reached whether ERIH should play a larger role, while the oligopoly of major publishing houses and their bibliometrics steadily enlarge their positions. New ways of open review ratings with self-publishing have stepped into the field. The rise of ResearchGate is but one example of an alternative scoring system based on a scholarly social network which, however, still faces the same problems of fair indexing (Murray 2014). How to change the conduct of social sciences and humanities and their reputation-based system towards a more data-based is still an open question. Neither the established reputation-based system nor a more quantitative combination of many indices is better, more abstract or more valuable. Fairness cannot be born from the head of computers and of scholarly networks alone.