Background and Introduction

To establish a specific research profile, the identification of strengths and weaknesses in research and teaching is imperative for a modern international university, especially in times of economic strain. There are various attempts described and documented in literature, e.g. by Fang and Casadevall (2016), Welpe et al. (2015), van Vught and Huisman (2014) and Ouchi (1979). With its 6700 academics, including more than 400 professors, the University of Vienna is the largest research and educational institution in Austria. The vast array of disciplinary and interdisciplinary research is organized in 20 Faculties covering common research fields (theology, law, business, economics and statistics, computer science, historical and philological studies, social sciences, psychology, life sciences, natural sciences, mathematics). During a periodical discussion process the scientists of the University of Vienna define a characteristic set of key-research areas for their faculty (bottom-up approach without quantitative analyses), e.g. “Quantum optics, quantum nanophysics and quantum information” (Faculty of Physics), “Governance, democracy, solidarity” (Faculty of Social Sciences) or “RNA biology” (Centre for Molecular Biology). 100 key-research areas are currently listed in the University’s development plan (the University’s main strategic document).Footnote 1 In order to adhere to its policy to cover a broad range of research areas and considering the institutional responsibility towards all academic fields the research profile of the University of Vienna can therefore not be restricted to a few selected fields. Instead it is necessary to identify unique characteristics (what the University of Vienna in particular stands for) and to decide which research areas need to be emphasised. A research profile based on these reflections is a valuable component for the university’s public image, for the correct adjustment of the self-perception, for resource decisions and benchmarking (Laudel and Weyer 2014; van Vught et al. 2010). The challenge for Universities is to find the right balance between broadness and focus (Müller 2017). In this context, the Rectorate started a project in 2016 to define interdisciplinary research specializations (IRS) which are the key subject of our analysis. The Austrian Federal Ministry of Education, Science and Research proposed the use of esteem indicators to assess IRS (Burkert 2016) in order to develop a strategic commitment to certain areas and to generate, trust and critical mass. Based on this proposal the Rector’s Office established a specific set of esteem indicators (see “Methodology” section) mainly based on competitive third party funding and science awards. Despite the well-known limitations of higher education rankings (Sorz et al. 2015; Bookstein et al. 2010; Hazelkorn 2007) results of subject rankings were also included. Nine IRS were identified using this approach. While this procedure found general acceptance within the University, bibliometric indicators for scientific accomplishment like publication output and citations, were not considered in the IRS defining process so far. There could be areas which researchers of the University excel, but were not considered yet as a part of a research specialisation, since there is no significant third party funding (due to either lack of activity or lack of available funding). This could be especially relevant for the Social Sciences and Humanities (SSH) that are well represented at the University of Vienna. In 2017, approximately 40% of the total full-time equivalent scientific staff at the University of Vienna could be attributed to the humanities and approximately 22% to the social sciences. This made it imperative to investigate possible gaps in coverage of the current esteem-factor approach. Furthermore, the inclusion of publication and citation data would give a better multidimensional picture of the University’s research strengths and could back-up the existing esteem indicators (García et al. 2013; Rafols et al. 2010). A comprehensive method to identify research specialisations is of increasing importance since in addition to their role in the public image of the University of Vienna, they are envisaged as potential basis of strategic resource decisions (“strengthening the strengths”). In this way, whether a subject area belongs to an existing or a potential new research specialisation could also have an influence on the dedication of professorships. Existing and new interdisciplinary research specialisations also could be promoted by announcing additional tenure track positions and by making investments in academic and spatial infrastructure. Furthermore, they can be the starting point for innovative interdisciplinary professorships. They can also provide a link between new (interdisciplinary) master’s programmes and thus, by means of research-led teaching, ensure young academics work in the areas of excellence at the University of Vienna.

This paper is an extended version of the proceedings paper presented at the 2019 ISSI (International Society for Scientometrics and Informetrics (Sorz et al. 2019). A new analysis considering two further case studies (IRS) has been included. All the sections, especially methodology, results and conclusions have been accordingly enhanced.

Aims of the study

The main research questions of this study are:

  1. 1.

    Does a bibliometric analysis confirm the established esteem-factor approach to define IRS?

  2. 2.

    Does the proposed set of bibliometric indicators complement the well-established esteem indicators?

  3. 3.

    Are there research areas, not covered by the esteem-factor approach that should be included in the IRS?

To answer questions 1–3 we compared the results of the esteem-factor approach (Approach 1) with a bibliometric analysis of high publication numbers, high publication numbers in top journals and high impact (Approach 2). For Approach 2 we used two alternative clustering methods to aggregate the publications. First, we used the subject categories as assigned in the Web of Science Core Collection (Clustering 1). As an alternative clustering (Clustering 2) we used a list of researchers associated with the esteem factors (awardees, grantees, and the researchers in their third-party-funded projects) of four IRS (two STEM and two SSH).

Methodology

Figure 1 summarizes the workflow of the two approaches used in this study.

Fig. 1
figure 1

Study design for Approaches 1 and 2 as described in the “Methodology” section

Full size image

Approach 1 (esteem-factors)

In order to determine the IRS we used the pre-defined 100 key research areas of the University’s Faculties and combined them with output indicators (esteem factors). The key-research areas have previously been defined by the researchers of the faculties in a “bottom-up” discussion process, without any accompanying scientometric analyses. As esteem-factors we considered only running projects and the most recent ranking results:

National grants/prizes

  • FWF-START-Grant: Grant by the largest Austrian Funding Agency (FWF) for outstanding young researchers of any discipline to establish a research group (800k-1.2 Mio. €).

  • Wittgenstein-Award: Highest endowed research prize in Austria (up to 1.5 Mio. €)

  • FWF-Special Research Programmes (SFB) for the establishment of interdisciplinary research networks (up to 1 Mio. € per year/for up to 8 years)

  • FWF-Doctoral Programmes (DK) provide funding for 5-20 PhD students for up to 8 years

  • WWTF-Grants: competitive grants provided by the Vienna Science and Technology Fund Projects, endowed chairs and Vienna Research Groups in various research fields (up to 1 Mio. Euro/project)

  • CD-Labs: application-oriented research funding, connecting business and science; research groups (5-15 people) up to 7 years (110 k-700 k/year; 50–60% public funding, 40–50% private funding by companies)

  • Laura-Bassi-Centres: Governmental funding for predominantly female research teams with a female head on the interface academia/industry (ca. 1 Mio./centre/year for up to 7 years)

International grants

  • ERC-Grants (Starting, Consolidator and Advanced Grants; Proof of Concept Gants)

  • Collaborative EU-Projects (FP7, Horizon2020) > 300 k€ (portion of the UoV)

  • ESFRI/ERIC-Infrastructures

  • Innovative Medicines Initiative (IMI) Projects

Higher education Rankings (TOP 200)

  • THE Ranking by subject

  • QS-Ranking by subject

  • ARWU (Shanghai) Subject Ranking

Additional factors

  • Special facilities with high national or international visibility established with University funding (e.g. the European Law Institute)

  • Endowed chairs/professorships (e.g. by the Simonis Foundation)

Based on the 100 we identified 37 key-research areas that feature at least one esteem-factor, thus at least one researcher involved with the key-research area was granted at least one of the above mentioned prizes/awards or the scientific field achieved a high position in the most recent ranking. This was done manually on researcher level, thus if a researcher of a faculty was awarded with a grant or project, it was associated with a key research area of the faculty that fit to the researchers profile. We used only ranking results that could be attributed to a key research area, e.g. “Ecology” but not e.g. “Arts and Humanities”. We aggregated the 37 key-research areas manually into nine IRS. We defined the IRS by clustering thematically or methodologically similar or adjacent key-research areas and named each IRS to provide an overarching theme over the included areas, e.g. IRS2 “Materials and the quantum level”, comprising of four key-research areas of two Faculties active in material science and quantum physics. The Rectorate and the University’s Scientific Advisory Board, consisting of international high-level researchers from various fields, approved the indicator set and the final clustering of the IRS, as suggested by the authors. No further quantitative indicators were applied in this step. The order in which we listed the research specialisations (IRS 1-9) follows the principle of going from the theoretical to the concrete and, in turn, going from the smallest to the biggest scale level (see “Results” section).

Approach 2 (bibliometric analysis)

For Approach 2 we used data from the Web of Science Core Collection (WoS CC) including the analytic tool InCites. As a starting point, we established three premises to decide whether a WoS category can be considered as a research strength:

  1. 1.

    High publication activity: measured by the number of publications indexed in international databases. We retrieved all documents indexed in WoS Core Collection between 2010 and 2017, affiliated to the University of Vienna and analysed them according to the Web of Science category and research areas. To consider different publication cultures we applied a threshold of at least 100 publications in the STEM fields and 60 for the Social Sciences and Arts & Humanities.

  2. 2.

    High performance in top journals according to the subject category: measured by the percentage of documents in Q1 journals. Quartiles are based on the journal impact factor thus defining Q1 as the top 25% (data from Journal Citation report, JCR). The JIF calculated for the JCR edition of the publication year was used for all analysed publications. This parameter can be very helpful to provide a significant statement for the more recent publications years where the citation window is still too short in comparison to the cited half-life (Gorraiz and Gumpenberger 2012; Gorraiz et al. 2017).

  3. 3.

    High citation impact: measured by three normalized citation indicators, to account for differences in citation cultures across research fields and the differences in cited and citing half-lives. The ‘Category Normalized Citation Impact’ (CNCI) and the percentage of publications in the Top 1% and Top 10% most cited publications of the corresponding WoS Category, in the same publication year (percentile scores). The CNCI provides the citation impact (citations per paper) normalized for subject, year and document type. A publication with a CNCI value of 1.20 is 20% above the expected citation rate in the corresponding WoS Category in the same publication year. A publication with a value of 0.80 is 20% below the world’s expected citation rate. For a collection of publications, the mean value of all CNCI values is calculated. However, the CNCI is again an average or mean value and does not consider the characteristic skewness of the citation distribution. The percentiles represent the citation count threshold for different percentile cuts for each field and year. For instance, the 10th percentile represents the number of the Top 10% most cited papers in the corresponding category and for the current publication year. Top 10% is considered as a measure of ‘excellence’. We calculated percentile scores according to the baselines published in InCites for the corresponding WoS Categories. However, the CNCI as well as the percentage documents in the Top 10% and Top 1% provides rather complementary information (Gorraiz and Gumpenberger 2015, 2016; Moed 2017).

We tried to match the esteem-based IRS (Approach 1) with bibliometric indicators (Approach 2) and opted for a “hard approach” including all three bibliometric impact indicators. Thus, we selected only categories as research strengths if the values of all three indicators exceed the total average value retrieved for all the publications of that institution and if they exceed their expected values: CNCI > 1, percentage of Top 10% and Top 1% higher than 10% and 1% respectively.

Furthermore, in Approach 2 we used two different clustering methods to aggregate the publications (Clustering 1, 2). For Clustering 1 we used the subject categories as assigned in the WoS Core Collection. For Clustering 2 we identified the scientists’ names associated with the esteem indicators of four research specializations (STEM: “Materials and the quantum level”, IRS2; “Molecules, cells and their interaction”, IRS3; SSH: “Construction of identity and concepts of society”, IRS6; and “Cognition, communication and systemic reflection”, IRS7) and retrieved their publications in WoS Core Collection for the time period 2010-2017. These were analysed in InCites in order to retrieve all the indicators for each involved WoS subject category. The identification of the scientists was performed according to the information recorded in the staff database of the University of Vienna.

Results

Approach 1

We identified nine IRS (IRS 1-9, with underlying key-research areas and awarded esteem-factors):

  • IRS 1: Models and algorithms

    Key-research areas: “Algorithms, Software and Computing Technologies”, “Analysis, geometric structures and mathematical physics”, “Arithmetic, algebra and discrete mathematics”, “Biomathematics and Dynamic Systems”, “Computational Sciences”, “Particle Physics, Gravitational Physics and Mathematical Physics”, “Stochastics und Financial Mathematics”, „Logic“. Esteem factors: 1 ERC, 4 FWF-START, 2 FWF-SFB, 1 FWF-DK, 1 EU-Projects, 2 IMI-Projects, 6 WWTF-Projects; 1 Berta-Karlik-Chair, 1 Chair by Simonis-Foundation;, QS-Ranking, ARWU-Ranking

  • IRS 2: Materials and the quantum level

    Key-research areas: “Functional and sustainable materials chemistry”, “Complex Nanoscale Matter”, “Quantum Optics, Quantum Nanophysics and Quantum Information”. Esteem factors: 4 ERC, 2 FWF-START, 2 FWF-SFB, 3 EU-Projects, 2 CD-Labs, ARWU-Ranking, QS-Ranking, THE Ranking

  • IRS 3: Molecules, cells and their interaction

    Key-research areas: „Chromosome‐Dynamics“, “Computational Chemistry and und Biomolecular Simulation”, „Integrative Structural Biology“, „RNA-Biology“, “Cell-Signalling”. Esteem factors: 7 ERC, 3 FWF-START, 4 FWF-DK, 5 FWF-SFB, 2 WWTF; 2 Berta-Karlik-Chairs), 1 Laura Bassi Centre, 1 CD-Lab, 3 WWTF-Projects (2014), 1 EU-Project, THE-Ranking,

  • IRS 4: Food and drugs

    Key-research areas: “Biological and medicinal chemistry”, “Drug Discovery from Nature”, “Food chemistry and Physiological chemistry”. Esteem factors: 3 ERC, 1 FWF-SFB, 1 FWF-DK, 1 WWTF-Project, 2 EU-Projects, 4 IMI-Projects, 3 CD-Labs,

  • IRS 5: Microbiology, ecosystems and evolution

    Key-research areas: “Evolution of Organismal Complexity”, “Microbial Ecology and Ecosystems”, “Patterns and Processes in Plant Evolution and Ecology”. Esteem factors: 8 ERC; 1 Wittgenstein-Award, 1 FWF-DK, 1 FWF- START, 1 EU-Project, 2 WWTF-Projects, THE-Ranking, Shanghai-Ranking

  • IRS 6: Construction of identity and concepts of society

    Key-research areas: “Community, Conflict, Integration”, „Global Cultures and Identities“, “Cultures of the Euro-Mediterranean region and antiquity studies”, “Migration, Citizenship and Belonging”, “Austrian politics and Europeanisation” Esteem factors: 7 ERC, 1 FWF-SFB, 1 FWF-START; 1 Wittgenstein-Award, “Austrian Centre for Digital Humanities”; 1 ESFRI-ERIC-Infrastructure, 1 WWTF-Project, QS-Ranking

  • IRS 7: Cognition, communication and systemic reflection

    Key-research areas: “Justification and critique of norms in ethics, law and politics”, “Cognitive Science and Neuroscience”, “Systemic and functional dimensions of communication“, „Technologies and cognitive processes in translation and multilingual language data processing”, “Theories of knowledge, of science and of the social”. Esteem factors: 6 ERC, 1 FWF-SFB, 3 FWF-DK, 1 FWF-START; 5 WWTF-Projects, 2 Berta-Karlik-Chairs, 1 ESFRI-ERIC-Infrastructure, 1 EU-Project, QS-Ranking, THE-Ranking, Shanghai-Ranking

  • IRS 8: Internationalisation of the economy and law

    Key-research areas: “Europeanisation of commercial law and business law”, “Changing markets and institutions”, “Fundamental rights and human rights in complex relations of power”. Esteem factors: 4 FWF-DK, 1 CD-Lab, 1 EU-Project, 1 Endowed Chair “Human Rights”, “The European Law Institute”; QS-Ranking, THE-Ranking;

  • IRS 9: The environment and cosmic processes

    Key-research areas: “Physics and the Environment”, “Cosmos”, “Environment”

    Esteem factors: 2 ERC, 1 FWF-NFN, 7 EU-Projects, 1 WWTF-Project, 4 Research Platforms, QS-Ranking

Approach 2

The results originating from the bibliometric analyses of Approach 2 are summarised in the following six tables. Table 1 shows the Pearson correlation between the selected indicators described in “Methodology” section for all the Web of Sciences categories in which the University of Vienna published between 2010 and 2017.

Table 1 Correlation between the four indicators according to their values in the 236 WoS subject categories where the University of Vienna published between 2010 and 2017
Full size table

Clustering 1: correlation of bibliometric indicators

The very low correlation values between the percentage of documents in Q1 journals and the three normalized citation counts can be explained by the fact that there are no Q1 journals in the Arts & Humanities. Table 2 provides a more detailed view of the relationship between visibility (percentage of documents in Q1 journals) and impact represented by the percentiles. We calculated correlations for the total period (2010 and 2017) as well as for the period between 2010 and 2014 in order to guarantee a larger citation window, which is of greater value in disciplines with longer half-lives of citations.

Table 2 Distribution of the percentage of documents in Q1 journals and the citation percentile for two periods: 2010–2017 and 2010–2014
Full size table

It shows that the percentage of documents in Q1 journals does not strongly correlate with the citation percentiles. Furthermore, almost half of the publications do not contain this quartile information because they were not published in journals or the publishing journals do not have an impact factor in JCR. Table 2 shows that the percentage of documents in Q1 journals in top percentiles increases considerably with the citation window. The median correlation between the CNCI and the percentage of documents in Top 10% reported in Table 1 underpins the simultaneous use of both indicators, due to the skewness of the citation distribution. The correlation is higher as expected between the CNCI and the percentage of documents in Top 1%, because the outliers (extremely highly cited publications) strongly shapes the CNCI.

Clustering 1: matching of WoS Subject categories with IRS

Table 3 shows all WoS Categories with exceeding values of the three selected impact indicators in comparison to the mean baseline for the overall university (baseline for all items) and with a threshold of at least 100 publications. We sorted them according to the descending percentage of documents in Top 10% most cited. As additional information, we have added the IRS—as identified in Approach 1—that would correspond thematically to the WoS category (last column). The authors did this manually by matching thematically categories of both approaches, e.g. WoS category “Computer Science, Theory and Methods”, was matched with IRS 1 “Models and algorithms”. Some of the Web of Science categories were ambiguous and thus matched with 2 or more IRS, e.g. “Physics, Multidisciplinary” with IRS 1 and IRS 2 (“Materials and the quantum level”). Some WoS were too unspecific to be matched, e.g. “Instruments and Instrumentation”.

Table 3 Top WoS categories according to the three selected impact indicators (CNCI, Top 10%, Top 1%) and sorted by the percentage of documents in Top 10%
Full size table

Table 4 shows again all WoS Categories with exceeding values of the three selected impact indicators as well as the percentage of documents in Q1 journals (also sorted according to the descending percentage of documents in Top 10% most cited).

Table 4 Top WoS categories according to the three selected impact indicators and the percentage of documents in Q1 journals for the STEM research areas
Full size table

The analysis shows that the inclusion of the percentage of documents in Q1 journals can be used as additional selection criterion to identify potential research strengths. Furthermore, it is in good agreement with the normalized citation indicators. In addition, the research strengths identified in Approach 1 largely match the results of the bibliometric analysis. However, there are some incongruities. Most strikingly, the subject category of “Mathematics” is missing in the bibliometric analysis. While IRS1 “Models and algorithms” comprises largely of key-research areas associated with the Mathematics faculty, this subject category does not show up in the bibliometric analysis. Several subject categories that can be attributed to “Mathematics”, score high in the Top 10% indicator, but fail to hit the predetermined thresholds for all three impact indicators (e.g. “Mathematical and Computational Biology”, “Mathematics, Applied” or “Mathematics”). See full list of WoS subject categories ranked by the percentage of documents in Top 10% exceeding the baseline for all items in “Appendix”. This shows that the use of the CNCI as an additional indicator may be detrimental to those categories that show lower citation densities. Our hard approach was a first attempt to identify institutional research strengths, but needs a deeper analysis in the future to correct such anomalies.

Furthermore, some fields showed up in the WoS categories that are not covered at the University of Vienna at the first sight, e.g. “Cardiac and Cardiovascular Systems” and “Surgery” do not match because the University has no Medical Faculty. Thus, they cannot be assigned to the IRS without further analysis, e.g. identification of the researchers publishing in these subject categories. “Physical Geography” can be associated with IRS9 and “Optics” with IRS 2, however there are no matching underlying key-research areas based on esteem-indicators in these field at the University. This could be examples for a research strengths defined only by bibliometric indicators.

Clustering 2: results for four IRS 2, IRS3, IRS 6 and IRS 7

For Clustering 2 the following four IRS were selected: “Materials and the quantum level” (IRS2), “Molecules, cells and their interaction” (IRS3), “Construction of identity and concepts of society” (IRS6) and “Cognition, communication and systemic reflection” (IRS7). We identified all scientists affiliated to the University of Vienna that were involved in these IRS and retrieved their publications for the period 2010–2017. IRS2 and IRS3 can be assigned to STEM, IRS6 to SSH and IRS7 is a mixed IRS, comprising of key-research areas of the Social Sciences, Psychology and Life Sciences. Tables 5, 6, 7 and 8 show the results for IRS2, IRS3, IRS6 and IRS 7 respectively. We sorted the WoS Categories according to the number of Web of Science Documents for all IRS considered.

Table 5 Results for IRS2 (“Materials and the quantum level”)
Full size table
Table 6 Results for IRS3 (“Molecules, cells and their interaction”)
Full size table
Table 7 Results for IRS6 (“Construction of identity and concepts of society”)
Full size table
Table 8 Results for IRS7 (“Cognition, communication and systemic reflection”)
Full size table

IRS2 and IRS3 (STEM): The dataset comprises of 50 scientists and 998 publications for IRS2. For IRS2 the values of all four applied indicators are at least twice as high as expected and considerably higher as for the whole University (compare baselines in Tables 5 and 3). For IRS3 the dataset comprises of 37 scientists and 716 publications. The values of all four applied indicators are again at least twice as high as expected and considerably higher as for the whole University (compare baselines in Tables 6 and 3). The main WoS categories responsible for the high impact in IRS2 are “Physics, Multidisciplinary” and “Optics”. In the case of “Physics, Multidisciplinary” the values for the IRS2 are twice as high as for the whole university and four times higher as the expected values (see Table 3). All WoS categories retrieved for IRS2 can be attributed to at least one key-research area allocated to IRS2. The main WoS categories responsible for the high impact of IRS3 are “Biochemical Research Methods”, “Immunology” and “Biology”. Also for IRS3, the subject categories retrieved from Web of Science match very well with the esteem-factor based results.

IRS6 (SSH): The dataset IRS6 consisted of 137 scientists and 1116 publications. In contrast to the STEM related IRS2 and IRS3, the figures for the impact indicators are much lower (Table 7). This is due to the WoS Category “Religion”, where almost 550 publications were book reviews, which have no impact. The main categories responsible for the high impact in IRS6 are “Political Science” and “Communication”. For IRS6 “Communication”, “Demography” and “Political Science” show high values of the three applied indicators (note: Political Science is not included in Table 3; CNCI lower than the baseline for the whole university 1.4; 15.4% and 1.3% in the Tops 10% and 1% respectively; see “Appendix”).

IRS7 (STEM/SSH): The dataset for IRS7 consisted of 64 scientists and 689 publications. The average scores for IRS7 are twice as high as expected. The main WoS categories responsible for the high impact are “Zoology”, “Biology” and “Neurosciences”. All the categories related to psychology fail to meet high scores in almost all four measures, except “Psychology, Multidisciplinary”. Especially the percentage of publications in Q1 journals is considerably lower than for IRS2 and IRS3 (topics related to the STEM) but higher than for IRS6 (related to SSH). The subject categories retrieved from Web of Science for IRS7 match very well with the esteem-factor based results and almost all WoS categories can be attributed to one or various key-research areas that amount to IRS7. Only the WoS category “Ophthalmology” does not match, since the University of Vienna does not have a Medical Faculty (see also results of overall analysis, where the fields “Surgery” and Cardiac and Cardiovascular Systems” where identified, Table 3).

The results additionally show how a subject category, for example Biology, can achieve different values within different research specialisations (compare the scores of Biology in Tables 4, 6 and 8). This cannot be discerned when considering the total performance of the University (see Table 3).

Discussion and conclusions

Our case study shows that research strengths of a University can successfully be assessed using only esteem-based indicators (Approach 1). The esteem based approach found wide approval within the University and its researchers, by University leadership and by its international Scientific Advisory Board. However, we also found research areas that could only be identified using bibliometric indicators, e.g. “Physical Geography”, which was not identified in Approach 1, but showed a high number of publications and scored above the predetermined thresholds for all bibliometric indicators (see Table 3). Using only bibliometric indicators (Approach 2) also showed limitations, e.g. not identifying the highly esteemed “Mathematics” area, which was ranked as #37 by the ARWU-Ranking in 2017, or the identification of subject categories not even present at the University of Vienna, e.g. “Surgery” (see Table 3). Thus, we proved that the esteem factor-approach (Approach 1), can be successfully complemented with an overlaying citation analysis of the research output (Approach 2) and a combination of both approaches is favourable for identifying research strengths.

Normalised citation counts are the most adequate bibliometric indicators for this purpose with the percentage of documents in the “Top 10% most cited” as the most significant one. However, the CNCI and the Top 1% can be helpful in addition to provide a more complete and accurate description of the citation impact. The simultaneous use of all three citation normalized counts in order to identify research strengths or specialisations is only recommended for some disciplines (especially STEM fields), covering only one aspect of the identification process. High scores in any of the three indicators should be analysed separately, as shown for some disciplines like Mathematics, or Social Sciences and Humanities. On the other hand, the use of a single indicator may be too vague and unreliable. When using bibliometric analyses to identify research specializations the selection of citation indicators (e.g. in our study the combination of several indicators, or e.g. just Top 10% most cited) strongly determines the focus of the resulting research specializations which should be considered beforehand.

The complementary use of the percentage of documents in Q1 journals is only suitable for STEM and to some extent the Social Sciences. In these fields, it can help to overcome the low significance of citation analysis for the most recent publications years, which is explicable by the extremely low time frame (Glänzel et al. 2016a; Donner 2018). This indicator cannot be applied to the Arts & Humanities due to the lack of appropriate data and the limited role of citations in scholarly communication reflected by journal and proceedings literature. Clarivate Analytics has so far remained true to the philosophy of Eugene Garfield and is still reluctant to calculate journal impact factors (JIF) in the Arts & Humanities based on data from the Web of Science Core Collection (Garfield 1972, 2005). However, other providers of alternative journal impact measures, like Elsevier, Scimago and CWTS, have skipped this restriction. They publish their editions of CiteScore, Source Normalized Impact per Paper (SNIP) or Scimago Journal Rank (SJR) on a yearly basis, which also include the Arts & Humanities. Therefore, the availability of alternative journal impact measures seems to be of great advantage to these disciplines, as they might turn out to be crucial for evaluative practices in so far unapproachable scientific terrain. The inappropriate use of journal impact measures in evaluative contexts was already seriously criticized in the hard sciences (Glänzel and Moed 2002). This holds all the more true for the Arts & Humanities.

The inherent limitations of journals-assignment-based classification systems, and especially the ones used in the Web of Science Core Collection, hamper the identification of research strengths and research specializations. Calculated on journal level, the corresponding WoS Subject Categories are not uniquely assigned, e.g. in “Biology” as shown in the section Results. Thus, the resulting overlap complicated the interpretation of the results. Furthermore, the classifications are either too general, like the ESI Categories, or too fine, like the WoS Subject Categories, to provide clear and sound information. Therefore, we consider the use of other classification systems in the future, like the modified Leuven-Budapest classification system (Glänzel et al. 2016b). We found that the greatest hindrance in the bibliometric approach lies in the inherent limitations of journals-assignment-based classification systems and the considerable time and efforts for more accurate researcher-based publication analyses.

This study has also some noticeable limitations, in so far as it only presents a case study for one single university and applying both methods of publication clustering for only four of the nine IRS. Data coverage of the WoS Core Collection is incomplete particularly in the Social Sciences and Humanities due to the low coverage of indexed monographic literature.

However, the results clearly show the benefits of using both esteem-based and bibliometric analyses together for the accurate definition of research strengths. They show mainly similar results but additionally reveal new potential IRS requiring further discussion or interpretation. Furthermore, the design used in this study helps to understand the limitations of each approach.

In the future, we plan to analyse all nine IRS accordingly and will argument for the use of both approaches simultaneous for future studies, probably also including alternative metrics for broader impact assessment in the Humanities (Moed 2017).

Although it is a case study for the University of Vienna, all data and information required for both approaches are available for other institutions. Therefore, this study design can serve as a reference for other universities, if their Rectorate and Scientific Advisory Boards are willing to work on it and are interested in the identification of their IRS.