Skip to main content
SpringerLink
Log in

Measuring Science: Basic Principles and Application of Advanced Bibliometrics

  • Chapter
Springer Handbook of Science and Technology Indicators

Part of the book series: Springer Handbooks ((SHB))

Abstract

We begin with a short history of measuring science and discuss how the Science Citation Index has revolutionized the quantitative study of science and created a strong application potential. After reviewing the rationale of bibliometric analysis, we present the basic principle of the bibliometric methodology, with complex citation networks as a starting point. We show that the two main pillars of advanced bibliometric methods, citation-based analysis and science mapping, are both reducible to one and the same principle. From this basic principle we deduce a set of main indicators, particularly for the assessment of research output and international impact. Important elements include new approaches for identifying fields and research themes on the basis of a publication-level rather than a journal-level network; publication and citation counting; normalization of citation measures; the use of indicators based on averages versus those based on citation distributions; and weighting procedures and statistical reliability. In this account of the state of the art of advanced bibliometrics , we highlight in particular the developments in our Leiden institute, given its long-standing, extensive, and broad experience.

The next part of this chapter deals with practical applications of indicators, particularly real-life examples of evaluation studies. We further discuss several crucial issues such as the use of journal impact factors and h-index; the relation between peer review judgment and bibliometric findings; definition and delimitation of fields; assignment of publications; the influence of open access; webometrics and altmetrics; ranking of universities; and general objections to bibliometric analysis.

The second main pillar of the advanced bibliometric methodology is the development of science maps. We discuss the basic elements and the construction of both citation-relation and word-relation science maps. Further, we present a method to combine the two main pillars: the integration of citation analysis in science maps. This combined citation analysis and science mapping can be used to explore research related to socioeconomic problems. Recently developed bibliometric instruments enable tunable mapping, which opens up new analytical opportunities in monitoring scientific research. Finally, we contend that bibliometric indicators and maps are not just evaluation tools for science policymakers, research managers, and individual researchers, but also powerful instruments in the study of science.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 299.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 379.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • A.F.J. van Raan: Measuring science. Capita selecta of current main issues. In: Handbook of Quantitative Science and Technology Research, ed. by H.F. Moed, W. Glänzel, U. Schmoch (Springer, Dordrecht 2004) pp. 19–50

    Chapter  Google Scholar 

  • A.F.J. van Raan: Citations, h-index, journal impact and rankings: Not all sorrow and misery. CWTS: A short history of measuring science. In: Perspectives on the Past: 50 Years of FSW, ed. by J. van Holsteyn, R. Mom, I. Smit, H. Tromp, G. Wolters (Biblioscope, Utrecht 2013) pp. 86–103

    Google Scholar 

  • A.F.J. van Raan: Advances in bibliometric analysis: research performance assessment and science mapping. In: Bibliometrics: Use and Abuse in the Review of Research Performance, Wenner-Gren International Series, Vol. 87, ed. by W. Blockmans, L. Engwall, D. Weaire (London, Portland 2014) pp. 17–28

    Google Scholar 

  • F. Narin: Evaluative Bibliometrics: The Use of Publication and Citation Analysis in the Evaluation of Scientific Activity (National Science Foundation, Washington D.C. 1976)

    Google Scholar 

  • A. de Candolle: Histoire des sciences et des savants depuis deux siècles (Fayard, Paris 1987), 1st edn. 1873, 2nd edn. 1885 (Genève/Basel, H. Georg)

    Google Scholar 

  • A.J. Lotka: The frequency distribution of scientific productivity, J. Wash. Acad. Sci. 16, 317–323 (1926)

    Google Scholar 

  • S.C. Bradford: Documentation (Crosby, London 1948)

    Google Scholar 

  • R.S. Daniel, C.M. Louttit: Professional Problems in Psychology (Prentice-Hall, New York 1953)

    Book  Google Scholar 

  • E. Garfield: Citation indexes for science: A new dimension in documentation through association of ideas, Science 122(3159), 108–111 (1955)

    Article  Google Scholar 

  • P.F. Wouters: The Citation Culture, Ph.D. Thesis (University of Amsterdam, Amsterdam 1999)

    Google Scholar 

  • D.J. de Solla Price: Little Science, Big Science (Yale Univ. Press, New Haven 1963)

    Book  Google Scholar 

  • D.J. de Solla Price: Networks of scientific papers, Science 149, 510–515 (1965)

    Article  Google Scholar 

  • R.K. Merton: The Matthew effect in science. The reward and communication systems of science are considered, Science 159(3810), 56–63 (1968)

    Article  Google Scholar 

  • D.J. de Solla Price: Science Since Babylon (Yale Univ. Press, New Haven 1975)

    Google Scholar 

  • D.J. de Solla Price: A general theory of bibliometric and other cumulative advantage processes, J. Am. Soc. Inf. Sci. 27(5/6), 292–306 (1976)

    Article  Google Scholar 

  • M.M. Kessler: Bibliographic coupling between scientific papers, Am. Doc. 14, 10–25 (1963)

    Article  Google Scholar 

  • H. Small: Co-citation in the scientific literature: a new measure of the relationship between two documents, J. Am. Soc. Inf. Sci. Technol. 24, 265–269 (1973)

    Article  Google Scholar 

  • S. Cole, J.R. Cole: Scientific output and recognition, Am. Sociol. Rev. 62, 377–390 (1967)

    Article  Google Scholar 

  • J.R. Cole, S. Cole: The Ortega hypothesis, Science 178, 368 (1972)

    Article  Google Scholar 

  • National Science Board: Science Indicators 1972 (Government Printing Office, Washington DC 1973)

    Google Scholar 

  • OECD: The Measurement of Scientific and Technological Activities. Frascati Manual (Organization for Economic Co-operation and Development (OECD), Paris 1963)

    Google Scholar 

  • E. Garfield: Citation analysis as a tool in journal evaluation, Science 178(4060), 471–479 (1972)

    Article  Google Scholar 

  • E. Garfield: Citation Indexing: Its Theory and Application in Science, Technology and Humanities (Wiley, New York 1979)

    Google Scholar 

  • G. Pinski, F. Narin: Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics, Inf. Process. Manag. 12(5), 297–312 (1976)

    Article  Google Scholar 

  • M.P. Carpenter, M. Cooper, F. Narin: Linkage between basic research literature and patents, Res. Manag. 23(2), 30–35 (1980)

    Google Scholar 

  • K.H. Chang: Evaluation and Survey of a Subfield of Physics: Magnetic Resonance and Relaxation Studies in the Netherlands, FOM-Report, Vol. 37175 (FOM, Utrecht 1975)

    Google Scholar 

  • C.J.G. Bakker: Elektronenmicroscopie in Nederland, FOM-Report, Vol. 43105 (FOM, Utrecht 1977)

    Google Scholar 

  • Y. Elkana, J. Lederberg, R.K. Merton, A. Thackray, H. Zuckerman (Eds.): Toward a Metric of Science: The Advent of Science Indicators (Wiley, New York 1978)

    Google Scholar 

  • D.D. Beaver, R. Rosen: Studies in scientific collaboration, 1: Professional origins of scientific co-authorship, Scientometrics 1, 65–84 (1978)

    Article  Google Scholar 

  • G.N. Gilbert: Measuring the growth of science- review of indicators of scientific growth, Scientometrics 1, 9–34 (1978)

    Article  Google Scholar 

  • M.J. Moravcsik: Phenomenology and models of growth of science, Res. Policy 4, 80–86 (1975)

    Article  Google Scholar 

  • J. Vláchy: Mobility in science. Bibliography of scientific career migration, field mobility, international academic circulation and brain drain, Scientometrics 1, 201–228 (1979)

    Article  Google Scholar 

  • H.F. Moed, W.J.M. Burger, J.G. Frankfort, A.F.J. van Raan: The use of bibliometric data for the measurement of university-research performance, Res. Policy 14(3), 131–149 (1985)

    Article  Google Scholar 

  • B.R. Martin, J. Irvine: Assessing basic research: Some partial indicators of scientific progress in radio astronomy, Res. Policy 12, 61–90 (1983)

    Article  Google Scholar 

  • D. Sullivan, D. Koester, D.H. White, R. Kern: Understanding rapid theoretical change in particle physics- a month-by-month co-citation analysis, Scientometrics 2, 309–319 (1980)

    Article  Google Scholar 

  • H.D. White, B.C. Griffith: Author cocitation – a literature measure of intellectual structure, J. Am. Soc. Inf. Sci. 32, 163–171 (1981)

    Article  Google Scholar 

  • K.W. McCain: Longitudinal author cocitation mapping – the changing structure of macroeconomics, J. Am. Soc. Inf. Sci. 35, 351–359 (1984)

    Article  Google Scholar 

  • S.D. Haitun: Stationary scientometric distributions. 1: Different approximations, Scientometrics 4, 89–104 (1982)

    Article  Google Scholar 

  • A. Schubert, W. Glänzel: Statistical reliability of comparisons based on the citation impact of scientometric publications, Scientometrics 5, 59–74 (1983)

    Article  Google Scholar 

  • B.C. Peritz: A classification of citation roles for the social sciences and related fields, Scientometrics 5, 303–312 (1983)

    Article  Google Scholar 

  • M.E.D. Koenig: Bibliometric indicators versus expert opinion in assessing research performance, J. Am. Soc. Inf. Sci. 34, 136–145 (1983)

    Article  Google Scholar 

  • A.L. Porter, D.E. Chubin: An indicator of cross-disciplinary research, Scientometrics 8, 161–176 (1985)

    Article  Google Scholar 

  • A.G. Heffner: Funded research, multiple authorship, and sub-authorship collaboration in 4 disciplines, Scientometrics 3(1), 5–12 (1981)

    Article  Google Scholar 

  • C. Fenichel: Online searching – measures that discriminate among users with different types of experiences, J. Am. Soc. Inf. Sci. 32(1), 23–32 (1981)

    Article  Google Scholar 

  • K. Pavitt: Patent statistics as indicators of innovative activities—possibilities and problems, Scientometrics 7(1/2), 77–99 (1985)

    Article  Google Scholar 

  • L.G. Soete, S.M.E. Wyatt: The use of foreign patenting as an internationally comparable science and technology output indicator, Scientometrics 5(1), 31–54 (1983)

    Article  Google Scholar 

  • M. Callon, S. Bauin, J.P. Courtial, W. Turner: From translation to problematic networks: An introduction to co-word analysis, Soc. Sci. Inf. 22, 191–235 (1983)

    Article  Google Scholar 

  • A. Rip, J.P. Courtial: Co-word maps of biotechnology—an example of cognitive scientometrics, Scientometrics 6, 381–400 (1984)

    Article  Google Scholar 

  • A.F.J. van Raan (Ed.): Handbook of Quantitative Studies of Science and Technology (North Holland, Amsterdam 1988)

    Google Scholar 

  • A. Schubert, T. Braun: Relative indicators and relational charts for comparative-assessment of publication output and citation impact, Scientometrics 9(5-6), 281–291 (1986)

    Article  Google Scholar 

  • T. Braun, W. Glänzel, A. Schubert: World flash on basic research—the newest version of the facts and figures on publication output and relative citation impact of 100 countries 1981–1985, Scientometrics 13(5-6), 181–188 (1988)

    Article  Google Scholar 

  • R.R. Braam, H.F. Moed, A.F.J. Van Raan: Mapping of science by combined co-citation and word analysis, I: Structural aspects, J. Am. Soc. Inf. Sci. 42, 233–251 (1991)

    Article  Google Scholar 

  • R.R. Braam, H.F. Moed, A.F.J. Van Raan: Mapping of science by combined co-citation and word analysis, II: Dynamical aspects, J. Am. Soc. Inf. Sci. 42, 252–266 (1991)

    Article  Google Scholar 

  • P.O. Seglen: The skewness of science, J. Am. Soc. Inf. Sci. 43, 628–638 (1992)

    Article  Google Scholar 

  • S. Arunachalam, R. Srinivasan, V. Raman: International collaboration in science—participation by the Asian giants, Scientometrics 30, 7–22 (1994)

    Article  Google Scholar 

  • F. Narin: Patent bibliometrics, Scientometrics 30, 147–155 (1994)

    Article  Google Scholar 

  • F. Narin, K.S. Hamilton, D. Olivastro: The increasing linkage between US technology and public science, Res. Policy 26, 317–330 (1997)

    Article  Google Scholar 

  • U. Schmoch: Tracing the knowledge transfer from science to technology as reflected in patent indicators, Scientometrics 26, 193–211 (1993)

    Article  Google Scholar 

  • A.F.J. van Raan: Fractal dimension of co-citations, Nature 347, 626 (1990)

    Article  Google Scholar 

  • P.O. Seglen: Causal relationship between article citedness and journal impact, J. Am. Soc. Inf. Sci. 45, 1–11 (1994)

    Article  Google Scholar 

  • H.F. Moed, T.N. van Leeuwen: Improving the accuracy of the Institute for Scientific Information's journal impact factors, J. Am. Soc. Inf. Sci. 46, 461–467 (1995)

    Article  Google Scholar 

  • H.F. Moed, T.N. van Leeuwen: Impact factors can mislead, Nature 381, 186 (1996)

    Article  Google Scholar 

  • A.F.J. van Raan, T.N. van Leeuwen: Identifying the Fields for Mapping RTD Excellence in Life Sciences. First Approach (European Commission, Brussels 2001), Contract COPO-CT-2001-00001

    Google Scholar 

  • Academic Ranking of World Universities: http://www.shanghairanking.com

  • Times Higher Education World University Rankings: https://www.timeshighereducation.com/world-university-rankings

  • L. Waltman, C. Calero-Medina, J. Kosten, E.C.M. Noyons, R.J.W. Tijssen, N.J. van Eck, T.N. van Leeuwen, A.F.J. van Raan, M.S. Visser, P. Wouters: The Leiden Ranking 2011/2012: data collection, indicators, and interpretation, J. Am. Soc. Inf. Sci. Technol. 63(12), 2419–2432 (2012)

    Article  Google Scholar 

  • Leiden Ranking 2016: http://www.leidenranking.com/information/indicators

  • QS Top Universities: https://www.topuniversities.com/university-rankings

  • Scimago Institutions Rankings: http://www.scimagoir.com

  • U-Multirank: http://www.umultirank.org

  • A.F.J. van Raan: Fatal Attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods, Scientometrics 62(1), 133–143 (2005)

    Article  Google Scholar 

  • J.E. Hirsch: An index to quantify an individual's scientific research output, Proc. Natl. Acad. Sci. USA 102(467), 16569–16572 (2005)

    Article  Google Scholar 

  • A.W. Harzing: The Publish or Perish Book. Your Guide to Effective and Responsible Citation Analysis (Tarma Software Research, Melbourne 2010)

    Google Scholar 

  • A.W. Harzing: Publish or Perish, http://harzing.com/resources/publish-or-perish (2007)

  • H. Etzkowitz, L. Leydesdorff: The dynamics of innovation: From national systems and “Mode 2” to a triple helix of university-industry-government relations, Res. Policy 29, 109–123 (2000)

    Article  Google Scholar 

  • T.A. Brooks: Evidence of complex citer motivations, J. Am. Soc. Inf. Sci. 37, 34–36 (1986)

    Article  Google Scholar 

  • M.H. MacRoberts, B.R. MacRoberts: Author motivation for not giving citing influences—a methodological note, J. Am. Soc. Inf. Sci. 39, 432–433 (1988)

    Article  Google Scholar 

  • M.H. MacRoberts, B.R. MacRoberts: Problems of citation analysis, Scientometrics 36, 435–444 (1996)

    Article  Google Scholar 

  • P. Vinkler: Comparative investigation of frequency and strength of motives toward referencing, the reference threshold model—comments on theories of citation?, Scientometrics 43, 107–127 (1998)

    Article  Google Scholar 

  • L. Bornmann, H.D. Daniel: What do citation counts measure? A review of studies on citing behavior, J. Documentation 64(1), 45–80 (2008)

    Article  Google Scholar 

  • A.F.J. van Raan: In matters of quantitative studies of science the fault of theorists is offering too little and asking too much, Scientometrics 43, 129–139 (1998)

    Article  Google Scholar 

  • T.F. Frandsen, J. Nicolaisen: Citation behavior: A large-scale test of the persuasion by name-dropping hypothesis, J. Assoc. Inf. Sci. Technol. 65(2), 1278–1284 (2017)

    Article  Google Scholar 

  • A.J. Nederhof, A.F.J. van Raan: Peer-review and bibliometric indicators of scientific performance—A comparison of cum laude doctorates with ordinary doctorates in physics, Scientometrics 11(5-6), 333–350 (1987)

    Article  Google Scholar 

  • A.J. Nederhof, A.F.J. van Raan: A validation-study of bibliometric indicators—The comparative performance of cum laude doctorates in chemistry, Scientometrics 17(5-6), 427–435 (1989)

    Article  Google Scholar 

  • D.F. Horrobin: The philosophical basis of peer review and the suppression of innovation, J. Am. Med. Assoc. 263, 1438–1441 (1990)

    Article  Google Scholar 

  • C. Wennerås, A. Wold: Nepotism and sexism in peer-review, Nature 387, 341–343 (1997)

    Article  Google Scholar 

  • The Wellcome Trust: Women and Peer Review. An Audit of the Wellcome Trust Decision-Making on Grants (The Wellcome Trust/PRISM, London 1997)

    Google Scholar 

  • H.W. Marsh, L. Bornmann, R. Mutz, H.D. Daniel, A. O'Mara: Gender effects in the peer reviews of grant proposals: A comprehensive meta-analysis comparing traditional and multilevel approaches, Rev. Educ. Res. 79(3), 1290–1326 (2009)

    Article  Google Scholar 

  • A.J. Nederhof: The validity and reliability of evaluation of scholarly performance. In: Handbook of Quantitative Studies of Science and Technology, ed. by A.F.J. van Raan (Elsevier/North-Holland, Amsterdam 1988) pp. 193–228

    Chapter  Google Scholar 

  • W. Glänzel: A bibliometric approach to social sciences, national research performances in 6 selected social science areas, 1990–1992, Scientometrics 35, 291–307 (1996)

    Article  Google Scholar 

  • D. Hicks: The difficulty of achieving full coverage of international social science literature and the bibliometric consequences, Scientometrics 44, 193–215 (1999)

    Article  Google Scholar 

  • H.F. Moed, M. Luwel, A.J. Nederhof: Towards research performance measurement in the humanities, Libr. Trends 50, 498–520 (2002)

    Google Scholar 

  • L. Butler, M.S. Visser: Extending citation analysis to non-source items, Scientometrics 66(2), 327–343 (2006)

    Article  Google Scholar 

  • A.J. Nederhof: Bibliometric monitoring of research performance in the social sciences and the humanities: A review, Scientometrics 66(1), 81–100 (2006)

    Article  Google Scholar 

  • A.J. Nederhof, T.N. van Leeuwen, A.F.J. van Raan: Highly cited non-journal publications in political science, economics and psychology: A first exploration, Scientometrics 83(2), 363–374 (2010)

    Article  Google Scholar 

  • A. Linmans: Why with bibliometrics the humanities does not need to be the weakest link. Indicators for research evaluation based on citations, library holdings, and productivity measures, Scientometrics 83(2), 337–354 (2010)

    Article  Google Scholar 

  • L. Egghe, R. Rousseau: The influence of publication delays on the observed aging distribution of scientific literature, J. Am. Soc. Inf. Sci. 51, 158–165 (2000)

    Article  Google Scholar 

  • E. Garfield: Premature discovery or delayed recognition –why? In: Essays of an Information Scientist, Vol. 4 (1980) pp. 488–493

    Google Scholar 

  • W. Glänzel, B. Schlemmer, B. Thijs: Better late than never? On the chance to become highly cited only beyond the standard time horizon, Scientometrics 58(3), 571–586 (2003)

    Article  Google Scholar 

  • H. Small, E. Greenlee: Citation context analysis of a co-citation cluster-recombinant DNA, Scientometrics 2(4), 277–301 (1980)

    Article  Google Scholar 

  • H. Small, E. Sweeney: Clustering the science citation index using co-citations, I: A comparison of methods, Scientometrics 7, 393–404 (1985)

    Article  Google Scholar 

  • H. Small, E. Sweeney, E. Greenlee: Clustering the science citation index using co-citations, II: Mapping science, Scientometrics 8, 321–340 (1985)

    Article  Google Scholar 

  • H. Small: Visualizing science by citation mapping, J. Am. Soc. Inf. Sci. 50, 799–813 (1999)

    Article  Google Scholar 

  • E. Noma: An improved method for analysing square scientometric transaction matrices, Scientometrics 4, 297–316 (1982)

    Article  Google Scholar 

  • Q. Zhou, L. Leydesdorff: The normalization of occurrence and co-occurrence matrices in bibliometrics using Cosine similarities and Ochiai coefficients, J. Assoc. Inf. Sci. Technol. 67(11), 2805–2814 (2016)

    Article  Google Scholar 

  • H.P.F. Peters, A.F.J. van Raan: Co-word-based science maps of chemical engineering, 1. Representations by direct multidimensional-scaling, Res. Policy 22(1), 23–45 (1993)

    Article  Google Scholar 

  • H.P.F. Peters, A.F.J. van Raan: Co-word-based science maps of chemical engineering, 2. Representations by combined clustering and multidimensional-scaling, Res. Policy 22(1), 47–71 (1993)

    Article  Google Scholar 

  • L. Waltman, N.J. van Eck: A new methodology for constructing a publication-level classification system of science, J. Am. Soc. Inf. Sci. Technol. 63(12), 2378–2392 (2012)

    Article  Google Scholar 

  • L. Waltman, N.J. van Eck: A smart local moving algorithm for large-scale modularity-based community detection, European Physical Journal B 86(11), 471 (2013)

    Article  Google Scholar 

  • N.J. van Eck, L. Waltman: CitNetExplorer: A new software tool for analyzing and visualizing citation networks, J. Informetrics 8(4), 802–823 (2014)

    Article  Google Scholar 

  • K.W. Boyack, R. Klavans: Creation of a highly detailed, dynamic, global model and map of science, J. Am. Soc. Inf. Sci. Technol. 65(4), 670–685 (2014)

    Article  Google Scholar 

  • CitNetExplorer: http://www.citnetexplorer.nl/home

  • N.J. van Eck, L. Waltman: Software survey: VOSviewer, a computer program for bibliometric mapping, Scientometrics 84(2), 523–538 (2010)

    Article  Google Scholar 

  • VOSviewer: http://vosviewer.com

  • L. Egghe: Theory and practise of the g-index, Scientometrics 69(1), 131–152 (2006)

    Article  Google Scholar 

  • K.W. Boyack, R. Klavans: Co-citation analysis, bibliographic coupling, and direct citation: Which citation approach represents the research front most accurately?, J. Am. Soc. Inf. Sci. Technol. 61, 2389–2404 (2010)

    Article  Google Scholar 

  • M. Rosvall, D. Axelsson, C.T. Bergstrom: The map equation, Eur. Phys. J. ST 178, 13–23 (2009)

    Article  Google Scholar 

  • M. Rosvall, C.T. Bergstrom: Multilevel compression of random walks on networks reveals hierarchical organization in large integrated systems, PLoS ONE 6(4), e18209 (2011)

    Article  Google Scholar 

  • L. Subelj, N.J. van Eck, L. Waltman: Clustering scientific publications based on citation relations: A systematic comparison of different methods, PLoS ONE 11(4), e0154404 (2016)

    Article  Google Scholar 

  • D.J. de Solla Price: Toward a model for science indicators. In: Toward a Metric of Science: The Advent of Science Indicators, ed. by Y. Elkana, J. Lederberg, R.K. Merton, A. Thackray, H. Zuckerman (John Wiley, New York 1978)

    Google Scholar 

  • J. Ziman: From parameters to portents—and back. In: Toward a Metric of Science: The Advent of Science Indicators, ed. by Y. Elkana, J. Lederberg, R.K. Merton, A. Thackray, H. Zuckerman (John Wiley, New York 1978)

    Google Scholar 

  • VSNU: Assessment of Research Quality: Chemistry and Chemical Engineering (VSNU, Utrecht 2002)

    Google Scholar 

  • Uppsala University: Quality and Renewal 2007: An Overall Evaluation of Research at Uppsala University (Uppsala University, Uppsala 2007)

    Google Scholar 

  • M. Olensky, M. Schmidt, N.J. van Eck: Evaluation of the citation matching algorithms of CWTS and iFQ in comparison to the Web of Science, J. Assoc. Inf. Sci. Technol. 67(10), 2550–2564 (2016)

    Article  Google Scholar 

  • L. Bornmann, L. Leydesdorff, R. Mutz: The use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limits, J. Informetr. 7(1), 158–165 (2013)

    Article  Google Scholar 

  • L. Waltman, N.J. van Eck: A systematic empirical comparison of different approaches for normalizing citation impact indicators, J. Informetr. 7(4), 833–849 (2013)

    Article  Google Scholar 

  • L. Waltman, N.J. van Eck, T.N. van Leeuwen, M.S. Visser, A.F.J. van Raan: Towards a new crown indicator: An empirical analysis, Scientometrics 87(3), 467–481 (2011)

    Article  Google Scholar 

  • L. Waltman, N.J. van Eck, T.N. van Leeuwen, M.S. Visser, A.F.J. van Raan: Towards a new crown indicator: Some theoretical considerations, J. Informetr. 5(1), 37–47 (2011)

    Article  Google Scholar 

  • T. Opthof, L. Leydesdorff: Caveats for the journal and field normalizations in the CWTS (“Leiden”) evaluations of research performance, J. Informetr. 4(3), 423–430 (2010)

    Article  Google Scholar 

  • N.J. van Eck, L. Waltman, A.F.J. van Raan, R.J.M. Klautz, W.C. Peul: Citation analysis may severely underestimate the impact of clinical research as compared to basic research, PLoS ONE 8(4), e62395 (2013)

    Article  Google Scholar 

  • Q. Wang, L. Waltman: Large-scale analysis of the accuracy of the journal classification systems of Web of Science and Scopus, J. Informetr. 10, 347–364 (2016)

    Article  Google Scholar 

  • H.F. Moed: Measuring contextual citation impact of scientific journals, J. Informetr. 4(3), 265–27726a (2010)

    Article  Google Scholar 

  • L. Leydesdorff, T. Opthof: Scopus's Source Normalized Impact per Paper (SNIP) versus a journal impact factor based on fractional counting of citations, J. Am. Soc. Inf. Sci. Technol. 61(11), 2365–2369 (2010)

    Article  Google Scholar 

  • L. Leydesdorff: The revised SNIP indicator of Elsevier's Scopus, J. Informetr. 7(4), 859–860 (2013)

    Article  Google Scholar 

  • L. Waltman, N.J. van Eck: Source normalized indicators of citation impact: An overview of different approaches and an empirical comparison, Scientometrics 96(3), 699–716 (2013)

    Article  Google Scholar 

  • L. Waltman: A review of the literature on citation impact indicators, J. Informetr. 10(2), 365–391 (2016)

    Article  Google Scholar 

  • G. Abramo, C.A. D'Angelo: A farewell to the MNCS and like size-independent indicators, J. Informetr. 10(2), 646–651 (2016)

    Article  Google Scholar 

  • G. Abramo, C.A. D'Angelo: A farewell to the MNCS and like size-independent indicators: Rejoinder, J. Informetr. 10(2), 679–683 (2016)

    Article  Google Scholar 

  • W. Glänzel, B. Thijs, K. Debackere: Productivity, performance, efficiency, impact—What do we measure anyway? Some comments on the paper “A farewell to the MNCS and like size-independent indicators” by Abramo and D'Angelo, J. Informetr. 10(2), 658–660 (2016)

    Article  Google Scholar 

  • G. Sivertsen: A welcome to methodological pragmatism, J. Informetr. 10(2), 664–666 (2016)

    Article  Google Scholar 

  • L. Bornmann, R. Haunschild: Efficiency of research performance and the glass researcher, J. Informetr. 10(2), 652–654 (2016)

    Article  Google Scholar 

  • R. Danell: Evaluating research organizations' contribution to science is not the same task as evaluating the performance of their scientists, J. Informetr. 10(2), 655–657 (2016)

    Article  Google Scholar 

  • M. Thelwall: Not dead, just resting: The practical value of per publication citation indicators, J. Informetr. 10(2), 667–670 (2016)

    Article  Google Scholar 

  • M. Zitt: Paving the way or pushing at open doors? A comment on Abramo and D'Angelo “Farewell to size-independent indicators”, J. Informetr. 10(2), 675–678 (2016)

    Article  Google Scholar 

  • J. Ruiz-Castillo: Research output indicators are not productivity indicators, J. Informetr. 10(2), 661–663 (2016)

    Article  Google Scholar 

  • L. Waltman, N.J. van Eck, M. Visser, P. Wouters: The elephant in the room: The problem of quantifying productivity in evaluative scientometrics, J. Informetr. 10(2), 671–674 (2016)

    Article  Google Scholar 

  • A.F.J. van Raan: Performance-related differences of bibliometric statistical properties of research groups: cumulative advantages and hierarchically layered networks, J. Am. Soc. Inf. Sci. Technol. 57(14), 1919–1935 (2006)

    Article  Google Scholar 

  • A.F.J. van Raan: Scaling rules in the science system: Influence of field-specific citation characteristics on the impact of research groups, J. Am. Soc. Inf. Sci. Technol. 59(4), 565–576 (2008)

    Article  Google Scholar 

  • L. Leydesdorff, T. Opthof: Normalization at the field level: Fractional counting of citations, J. Informetr. 4(4), 644–646 (2010)

    Article  Google Scholar 

  • L. Leydesdorff, L. Bornmann: How fractional counting of citations affects the impact factor: Normalization in terms of differences in citation potentials among fields of science, J. Am. Soc. Inf. Sci. Technol. 62(2), 217–229 (2011)

    Article  Google Scholar 

  • J. Ruiz-Castillo, L. Waltman: Field-normalized citation impact indicators using algorithmically constructed classification systems of science, J. Informetr. 9(1), 102–117 (2015)

    Article  Google Scholar 

  • L. Waltman, N.J. van Eck: Field-normalized citation impact indicators and the choice of an appropriate counting method, J. Informetr. 9(4), 872–894 (2015)

    Article  Google Scholar 

  • R. van Noorden: Love thy lab neighbour, Nature 468, 1011 (2010)

    Article  Google Scholar 

  • K. Lee, J.S. Brownstein, R.G. Mills, I.S. Kohane: Does collocation inform the impact of collaboration?, PLoS ONE 5(12), e14279 (2010)

    Article  Google Scholar 

  • A. Gazni, C.R. Sugimoto, F. Didegah: Mapping world scientific collaboration: Authors, institutions, and countries, J. Am. Soc. Inf. Sci. Technol. 63(2), 323–335 (2012)

    Article  Google Scholar 

  • P. Zhou, R.J.W. Tijssen, L. Leydesdorff: University-industry collaboration in China and the USA: A bibliometric comparison, PLoS ONE 11(11), e0165277 (2016)

    Article  Google Scholar 

  • A.F.J. van Raan: On growth, ageing, and fractal differentiation of science, Scientometrics 47(2), 347–362 (2000)

    Article  Google Scholar 

  • D. Hicks, P. Wouters, L. Waltman, S. de Rijcke, I. Rafols: The Leiden Manifesto for research metrics, Nature 520(7548), 429–431 (2015)

    Article  Google Scholar 

  • A.F.J. van Raan: Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises, Scientometrics 36, 397–420 (1996)

    Article  Google Scholar 

  • A.F.J. van Raan: Scientometrics: State-of-the-Art, Scientometrics 38, 205–218 (1997)

    Article  Google Scholar 

  • Wikipedia: Retraction Watch, https://en.wikipedia.org/wiki/Retraction_Watch

  • J.A. Byrne, C. Labbé: Striking similarities between publications from China describing single gene knockdown experiments in human cancer cell lines, Scientometrics 110(3), 1471–1493 (2017)

    Article  Google Scholar 

  • T.N. van Leeuwen, M. Luwel: An in-depth analysis of papers retracted in the Web of Science. In: Proc. 19th Int. Conf. Sci. Technol. Indic., Leiden (2014) pp. 337–344

    Google Scholar 

  • A.F.J. van Raan, T.N. van Leeuwen, M.S. Visser: Severe language effect in university rankings: particularly Germany and France are wronged in citation-based rankings, Scientometrics 88(2), 495–498 (2011)

    Article  Google Scholar 

  • A.F.J. van Raan, T.N. van Leeuwen, M.S. Visser: Non-English papers decrease rankings, Nature 469(7328), 34 (2011)

    Article  Google Scholar 

  • A.F.J. van Raan: Sleeping beauties in science, Scientometrics 59(3), 461–466 (2004)

    Google Scholar 

  • T. Braun, W. Glänzel, A. Schubert: On sleeping beauties, princes and other tales of citation distributions, Res. Eval. 19(3), 195–202 (2010)

    Article  Google Scholar 

  • C. Lachance, V. Larivière: On the citation lifecycle of papers with delayed recognition, J. Informetr. 8, 863–872 (2014)

    Article  Google Scholar 

  • A.F.J. van Raan: Dormitory of physical and engineering sciences: Sleeping beauties may be sleeping innovations, PLoS ONE 10(10), e0139786 (2015)

    Article  Google Scholar 

  • P. Gorry, P. Ragouet: “Sleeping beauty” and her restless sleep: Charles Dotter and the birth of interventional radiology, Scientometrics 107(2), 773–784 (2016)

    Article  Google Scholar 

  • A.F.J. van Raan: Sleeping beauties cited in patents: Is there also a dormitory of inventions?, Scientometrics 110(3), 1123–1156 (2017)

    Article  Google Scholar 

  • Q.L. Burrell: Are “sleeping beauties” to be expected?, Scientometrics 65(3), 381–389 (2005)

    Article  Google Scholar 

  • J. Wang, F. Ma, M. Chen, Y. Rao: Why and how can sleeping beauties be awakened?, Electron. Libr. 30(1), 5–18 (2012)

    Article  Google Scholar 

  • B. van Calster: It takes time: a remarkable example of delayed recognition, J. Am. Soc. Inf. Sci. Technol. 63(11), 2341–2344 (2012)

    Article  Google Scholar 

  • J. Li, F.Y. Ye: The phenomenon of all-elements-sleeping-beauties in scientific literature, Scientometrics 92(3), 795–799 (2012)

    Article  Google Scholar 

  • J. Li, D. Shi, S.X. Zhao, F.Y. Ye: A study of the “heartbeat spectra” for “sleeping beauties”, J. Informetr. 8, 493–502 (2014)

    Article  Google Scholar 

  • J. Li: Citation curves of “all-elements-sleeping-beauties”: “flash in the pan” first and then “delayed recognition”, Scientometrics 100(2), 595–601 (2014)

    Article  Google Scholar 

  • G.A. Ronda-Pupo, J.S. Katz: The scaling relationship between citation-based performance and coauthorship patterns in natural sciences, J. Assoc. Inf. Sci. Technol. 68(5), 1257–1265 (2017)

    Article  Google Scholar 

  • T.N. van Leeuwen: Discussing some basic critique on journal impact factors: Revision of earlier comments, Scientometrics 92(2), 443–455 (2012)

    Article  Google Scholar 

  • A.F.J. van Raan: Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups, Scientometrics 67(3), 491–502 (2006)

    Article  Google Scholar 

  • L. Bornmann, R. Mutz, H.D. Daniel: Are there better indices for evaluation purposes than the h-index? A comparison of nine different variants of the h-index using data from biomedicine, J. Am. Soc. Inf. Sci. Technol. 59(5), 830–837 (2008)

    Article  Google Scholar 

  • L. Waltman, N.J. Van Eck: The inconsistency of the h-index, J. Am. Soc. Inf. Sci. Technol. 63(2), 406–415 (2012)

    Article  Google Scholar 

  • E.J. Rinia, T.N. Van Leeuwen, H.G. van Vuren, A.F.J. van Raan: Comparative analysis of a set of bibliometric indicators and central peer review criteria. Evaluation of condensed matter physics in the Netherlands, Res. Policy 27, 95–107 (1998)

    Article  Google Scholar 

  • E.J. Rinia, T.N. van Leeuwen, H.G. van Vuren, A.F.J. van Raan: Influence of interdisciplinarity on peer-review and bibliometric evaluations, Res. Policy 30, 357–361 (2001)

    Article  Google Scholar 

  • H.F. Moed: Citation Analysis in Research Evaluation (Springer, Dordrecht 2005) pp. 229–257

    Google Scholar 

  • L. Bornmann, H.D. Daniel: Selecting scientific excellence through committee peer review—A citation analysis of publications previously published to approval or rejection of post-doctoral research fellowship applicants, Scientometrics 68(3), 427–440 (2006)

    Article  Google Scholar 

  • Y. Gargouri, C. Hajjem, V. Larivière, Y. Gingras, L. Carr, T. Brody, S. Harnad: Self-selected or mandated, open access increases citation impact for higher quality research, PLoS ONE 5(10), e13636 (2010)

    Article  Google Scholar 

  • Dimensions database: https://app.dimensions.ai/discover/publication

  • H.F. Moed: The effect of “Open access” on citation impact: An analysis of ArXiv's condensed matter section, J. Am. Soc. Inf. Sci. Technol. 58(13), 2047–2054 (2007)

    Article  Google Scholar 

  • A. Kastberg: Is open access really good for science?, Europhys. News 45(2), 32 (2014)

    Google Scholar 

  • L. Björneborn, P. Ingwersen: Perspectives of webometrics, Scientometrics 50, 65–82 (2001)

    Article  Google Scholar 

  • M. Thelwall, A. Smith: Interlinking between Asia-Pacific University Web sites, Scientometrics 55, 363–376 (2002)

    Article  Google Scholar 

  • M. Thelwall, G. Harries: The connection between the research of a university and counts of links to its web pages: An investigation based upon a classification of the relationships of pages to the research of the host university, J. Am. Soc. Inf. Sci. 54, 594–602 (2003)

    Article  Google Scholar 

  • I.F. Aguillo: Building web indicators for the EU OA repository. In: Workshop on New Research Lines in Informetrics, IPP-CCHS (CSIC), Madrid (2011), http://digital.csic.es/bitstream/10261/40279/1/OpenAIRE%20Webometrics.pdf

    Google Scholar 

  • J. Priem, D. Taraborelli, P. Groth, C. Neylon: Altmetrics: A manifesto, http://altmetrics.org/manifesto/ (2010)

  • J. Priem, P. Groth, D. Taraborelli: The altmetrics collection, PLoS ONE 7(11), e48753 (2012)

    Article  Google Scholar 

  • J. Bar-Ilan: Data collection methods on the web for informetric purposes—A review and analysis, Scientometrics 50, 7–32 (2001)

    Article  Google Scholar 

  • C. Schlögl, J. Gorraiz: Global usage versus global citation metrics: The case of pharmacology journals, J. Am. Soc. Inf. Sci. Technol. 62(1), 161–170 (2011)

    Article  Google Scholar 

  • C. Schlögl, J. Gorraiz, C. Gumpenberger, K. Jack, P. Kraker: Comparison of downloads, citations and readership data for two information systems journals, Scientometrics 101(2), 1113–1128 (2014)

    Article  Google Scholar 

  • J. Bar-Ilan, S. Haustein, I. Peters, J. Priem, H. Shema, J. Terliesner: Beyond citations: Scholars' visibility on the social web. In: Proc. 17th Int. Conf. Sci. Technol. Indic., Montréal (2012) pp. 98–109

    Google Scholar 

  • H. Shema, J. Bar-Ilan, M. Thelwall: Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics, J. Assoc. Inf. Sci. Technol. 65(5), 1018–1027 (2014)

    Article  Google Scholar 

  • M. Thelwall, S. Haustein, V. Larivière, C. Sugimoto: Do altmetrics work? Twitter and ten other candidates, PLoS ONE 8(5), e64841 (2013)

    Article  Google Scholar 

  • Research Trends: Special issue on alternative metrics, Issue 37, June (2014)

    Google Scholar 

  • R. Costas, Z. Zahedi, P. Wouters: Do altmetrics correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective, J. Assoc. Inf. Sci. Technol. 66(10), 2003–2019 (2014)

    Article  Google Scholar 

  • P. Sud, M. Thelwall: Evaluating altmetrics, Scientometrics 98(2), 1131–1143 (2014)

    Article  Google Scholar 

  • C. Chorus, L. Waltman: A large-scale analysis of impact factor biased journal self-citations, PLoS ONE 11(8), e0161021 (2016)

    Article  Google Scholar 

  • Wikipedia: San Francisco Declaration on Research Assessment, https://en.wikipedia.org/wiki/San_Francisco_Declaration_on_Research_Assessment

  • L. Waltman, V.A. Traag: Use of the journal impact factor for assessing individual articles need not be wrong, https://arxiv.org/abs/1703.02334 (2017)

  • M. Zitt, H. Small: Modifying the journal impact factor by fractional citation weighting: The audience factor, J. Am. Soc. Inf. Sci. Technol. 59(11), 1856–1860 (2008)

    Article  Google Scholar 

  • L. Waltman, N.J. van Eck: The relation between Eigenfactor, audience factor, and influence weight, J. Am. Soc. Inf. Sci. Technol. 61(7), 1476–1486 (2010)

    Article  Google Scholar 

  • CWTS Journal Indicators: http://www.journalindicators.com/

  • V.P. Guerrero-Botea, F. Moya-Anegón: A further step forward in measuring journals' scientific prestige: The SJR2 indicator, J. Informetr. 6, 674–688 (2012)

    Article  Google Scholar 

  • Scimago journal & country rank: http://www.scimagojr.com/journalrank.php

  • Elsevier: CiteScore, https://www.elsevier.com/editors-update/story/journal-metrics/citescore-a-new-metric-to-help-you-choose-the-right-journal

  • F. Franceschini, D. Maisano, L. Mastrogiacomo: The museum of errors/horrors in Scopus, J. Informetr. 10(1), 174–182 (2016)

    Article  Google Scholar 

  • W.J.N. Meester, L. Colledge, E.E. Dyas: A response to “The museum of errors/horrors in Scopus” by Franceschini et al, J. Informetr. 10(1), 569–570 (2016)

    Article  Google Scholar 

  • S. Brin, L. Page: The anatomy of a large-scale hypertextual web search engine, Comput. Netw. ISDN Syst. 30(1–7), 107–117 (1998)

    Article  Google Scholar 

  • C.T. Bergstrom: Eigenfactor: Measuring the value and prestige of scholarly journals, Coll. Res. Libr. News 68(5), 314–316 (2007)

    Article  Google Scholar 

  • B. Zogala-Siudem, G. Siudem, A. Cena, M. Gagolewski: Agent-based model for the h-index—exact solution, Eur. Phys. J. B 89, 21 (2016)

    Article  Google Scholar 

  • Research Policy Team HEFCE: REF Independent review of the role of metrics in research assessment. Summary of responses submitted to the call for evidence, http://dera.ioe.ac.uk/21370/2/Summary%20of%20responses%20submitted%20to%20the%20call%20for%20evidence.pdf (Oct. 2014)

  • H.F. Moed, J. Bar-Ilan, G. Halevi: A new methodology for comparing Google Scholar and Scopus, J. Informetr. 10(2), 533–551 (2016)

    Article  Google Scholar 

  • R. van. Noorden: Science research engine links papers to grants and patents. The Dimensions database promises a financial perspective on scholarly literature, https://www.nature.com/articles/d41586-018-00688-0 (16 Jan. 2018)

  • D.M. Nichols, M.B. Twidale: Metrics for openness, J. Assoc. Inf. Sci. Technol. 68(4), 1048–1060 (2016)

    Article  Google Scholar 

  • Webometrics Ranking of World Universities: http://www.webometrics.info/en

  • M. Thelwall, K. Kousha: ResearchGate articles: Age, discipline, audience size, and impact, J. Assoc. Inf. Sci. Technol. 68(2), 468–479 (2017)

    Article  Google Scholar 

  • PLoS ONE: Altmetrics, http://journals.plos.org/plosone/browse/altmetrics

  • Oxford University Press: https://academic.oup.com/journals/pages/authors/article_level_metrics

  • Altmetrics: http://altmetrics.org

  • Impactstory: https://profiles.impactstory.org/

  • R. Haunschild, L. Bornmann: Normalization of Mendeley reader counts for impact assessment, J. Informetr. 10(1), 62–73 (2016)

    Article  Google Scholar 

  • M. Thelwall, P. Sud: Mendeley readership counts: An investigation of temporal and disciplinary differences, J. Assoc. Inf. Sci. Technol. 67(12), 3036–3050 (2016)

    Article  Google Scholar 

  • National Information Standards Organization (NISO): NISO Alternative Assessment Metrics (Altmetrics) Initiative, https://www.niso.org/standards-committees/altmetrics

  • The Scientist: Top 10 Retractions of 2016, http://www.the-scientist.com/?articles.view/articleNo/47813/title/Top-10-Retractions-of-2016/

  • H.F. Moed: A critical comparative analysis of five world university rankings, Scientometrics 110(2), 967–990 (2017)

    Article  Google Scholar 

  • M. Dobrota, M. Bulajic, L. Bornmann, V. Jeremic: A new approach to the QS university ranking using the composite i-distance indicator: Uncertainty and sensitivity analyses, J. Assoc. Inf. Sci. Technol. 67(1), 200–211 (2016)

    Article  Google Scholar 

  • A. Perianes-Rodriguez, J. Ruiz-Castillo: University citation distributions, J. Assoc. Inf. Sci. Technol. 67(11), 2790–2804 (2016)

    Article  Google Scholar 

  • C. Daraio, A. Bonaccorsi: Beyond university rankings? Generating new indicators on universities by linking data in open platforms, J. Assoc. Inf. Sci. Technol. 68(2), 508–529 (2017)

    Article  Google Scholar 

  • Committee on Publication Ethics (COPE): Promoting integrity in research and its publication, http://publicationethics.org/

  • E. Delgado Lopez-Cozar, N. Robinson-Garcia, D. Torres-Salinas: The Google Scholar experiment: How to index false papers and manipulate bibliometric indicators, J. Assoc. Inf. Sci. Technol. 65(3), 446–454 (2014)

    Article  Google Scholar 

  • L. Butler: Modifying publication practices in response to funding formulas, Res. Eval. 17, 39–46 (2003)

    Article  Google Scholar 

  • S. De Rijcke, P.F. Wouters, A.D. Rushforth, T.P. Franssen, B. Hammarfelt: Evaluation practices and effects of indicator use—A literature review, Res. Eval. 25(2), 161–169 (2016)

    Article  Google Scholar 

  • V. Larivière, R. Costas: How many is too many? On the relationship between research productivity and impact, PLoS ONE 11(9), e0162709 (2016)

    Article  Google Scholar 

  • N. Caplar, S. Tacchella, S. Birrer: Quantitative evaluation of gender bias in astronomical publications from citation counts, Nat. Astron. 1(6), 0141 (2017)

    Article  Google Scholar 

  • J. Wilsdon, L. Allen, E. Belfiore, P. Campbell, S. Curry, S. Hill, R. Jones, R. Kain, S. Kerridge, M. Thelwall, J. Tinkler, I. Viney, P. Wouters, J. Hill, B. Johnson: The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management (HEFCE, London 2015)

    Book  Google Scholar 

  • A. Ancaiani, A.F. Anfossi, A. Barbara, S. Benedetto, B. Blasi, V. Carletti, T. Cicero, A. Ciolfi, F. Costa, G. Colizza, M. Costantini, F. di Cristina, A. Ferrara, R.M. Lacatena, M. Malgarini, I. Mazzotta, C.A. Nappi, S. Romagnosi, S. Sileoni: Evaluating scientific research in Italy: The 2004–10 research evaluation exercise, Res. Eval. 24, 242–255 (2015)

    Article  Google Scholar 

  • A. Molinié, G. Bodenhausen: Bibliometrics as weapons of mass citation, Chimia 64, 78–89 (2010)

    Article  Google Scholar 

  • R.R. Ernst: The follies of citation indices and academic ranking lists. A brief commentray to ‘Bibliometrics as weapons of mass citation', Chimia 64, 90 (2010)

    Article  Google Scholar 

  • Wikipedia: Ludolph van Ceulen, https://en.wikipedia.org/wiki/Ludolph_van_Ceulen

  • E. Noyons, C. Calero Medina: Results of the bibliometric study on the Faculty of Veterinary Medicine of the Utrecht University 2001-2011, CWTS Report April 2012, Center for Science Technol. Studies (CWTS) (Leiden University, Leiden 2012), https://www.uu.nl/en/organisation/faculty-of-veterinary-medicine/veterinary-research/organisation/assessment-of-research-quality

    Google Scholar 

  • R. Costas, T.N. van Leeuwen, A.F.J. van Raan: Is scientific literature subject to a 'sell-by-date'? A general methodology to analyze the 'durability' of scientific documents, J. Am. Soc. Inf. Sci. Technol. 61(2), 329–339 (2010)

    Article  Google Scholar 

  • R. Costas, T.N. van Leeuwen, A.F.J. van Raan: The “Mendel syndrome” in science: Durability of scientific literature and its effects on bibliometric analysis of individual scientists, Scientometrics 89(1), 177–205 (2011)

    Article  Google Scholar 

  • R. Costas, T.N. van Leeuwen, A.F.J. van Raan: Effects of the durability of scientific literature at the group level: Case study of chemistry research groups in the Netherlands, Res. Policy 42(4), 886–894 (2013)

    Article  Google Scholar 

  • A.F.J. van Raan, T.N. van Leeuwen: Assessment of the scientific basis of interdisciplinary, applied research: Application of bibliometric methods in nutrition and food research, Res. Policy 31, 611–632 (2002)

    Article  Google Scholar 

  • CWTS B.V.: Monitoring & Evaluation, Leiden University, http://www.cwtsbv.nl/Monitoring-Evaluation

  • M. Thelwall, P. Sud: National, disciplinary and temporal variations in the extent to which articles with more authors have more impact: Evidence from a geometric field normalised citation indicator, J. Informetr. 10(1), 48–61 (2016)

    Article  Google Scholar 

  • W. Glänzel, M. Meyer: Patents cited in the scientific literature: An exploratory study of ‘reverse' citation relations, Scientometrics 58, 415–428 (2003)

    Article  Google Scholar 

  • A.F.J. van Raan: Patent citations analysis and its value in research evaluation: A review and a new approach to map technology-relevant research, J. Data Inf. Sci. 2(1), 13–50 (2017)

    Google Scholar 

  • B. Uzzi, S. Mukherjee, M. Stringer, B. Jones: Atypical combinations and scientific impact, Science 342, 468–471 (2013)

    Article  Google Scholar 

  • A. Thor, W. Marx, L. Leydesdorff, L. Bornmann: Introducing CitedReferencesExplorer (CRExplorer): A program for reference publication year spectroscopy with cited references standardization, J. Informetr. 10(2), 503–515 (2016)

    Article  Google Scholar 

  • G. Chen, L. Xiao: Selecting publication keywords for domain analysis in bibliometrics: A comparison of three methods, J. Informetr. 10(1), 212–213 (2016)

    Article  Google Scholar 

  • K.H. Lee, S.Y. Kim, H.J. Kim, M. Song: Comparative evaluation of bibliometric content networks by tomographic content analysis: An application to Parkinson's disease, J. Assoc. Inf. Sci. Technol. (2017), https://doi.org/10.1002/asi.23752

    Article  Google Scholar 

  • L. Leydesdorff, A. Nerghes: Co-word maps and topic modeling: A comparison using small and medium-sized corpora (N < 1000), J. Assoc. Inf. Sci. Technol. 68(4), 1024–1035 (2017)

    Article  Google Scholar 

  • A. Suominen, H. Toivanen: Map of science with topic modeling: Comparison of unsupervised learning and human-assigned subject classification, J. Assoc. Inf. Sci. Technol. 67(10), 2464–2476 (2016)

    Article  Google Scholar 

  • B. Wen, E. Horlings, M. van der Zouwen, P. van den Besselaar: Mapping science through bibliometric triangulation: An experimental approach applied to water research, J. Assoc. Inf. Sci. Technol. 68(3), 724–738 (2017)

    Article  Google Scholar 

  • E.C.M. Noyons, A.F.J. van Raan: Monitoring scientific developments from a dynamic perspective: Self-organized structuring to map neural network research, J. Am. Soc. Inf. Sci. 49(1), 68–81 (1998)

    Google Scholar 

  • L. Waltman, A.F.J. van Raan, S. Smart: Exploring the relationship between the engineering and physical sciences and the health and life sciences by advanced bibliometric methods, PLoS ONE 9(10), e111530 (2014)

    Article  Google Scholar 

  • L. Leydesdorff, I. Rafols: A global map of science based on the ISI subject categories, J. Am. Soc. Inf. Sci. Technol. 60(2), 348–362 (2009)

    Article  Google Scholar 

  • L. Leydesdorff, S. Carley, I. Rafols: Global maps of science based on the new Web-of-Science categories, Scientometrics 94(2), 589–593 (2013)

    Article  Google Scholar 

  • H. Small, K.W. Boyack, R. Klavans: Identifying emerging topics in science and technology, Res. Policy 43, 1450–1467 (2014)

    Article  Google Scholar 

  • D. Zhao, A. Strotmann: Can citation analysis of web publications better detect research fronts?, J. Am. Soc. Inf. Sci. Technol. 58(9), 1285–1302 (2007)

    Article  Google Scholar 

  • S. Jensen, X. Liub, Y. Yuc, S. Milojevic: Generation of topic evolution trees from heterogeneous bibliographic networks, J. Informetr. 10, 606–621 (2016)

    Article  Google Scholar 

  • G. Holton: Can science be measured? In: Toward a metric of science: the advent of science indicators, ed. by Y. Elkana, J. Lederberg, R.K. Merton, A. Thackray, H. Zuckerman (John Wiley, New York 1978)

    Google Scholar 

Download references

Acknowledgements

The author thanks Nees-Jan van Eck for the construction of the publication-level citation-based network map and the citation density neurology map. The text parts on the methodology of the Leiden Ranking, the CitNetExplorer, and the VOSviewer are largely based on the descriptions in the relevant CWTS webpages by Ludo Waltman and Nees-Jan van Eck.

Author information

Authors and Affiliations

  1. Centre for Science and Technology Studies (CWTS), Leiden University, Leiden, The Netherlands

    Anthony van Raan

Authors
  1. Anthony van Raan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anthony van Raan .

Editor information

Editors and Affiliations

  1. ECOOM and Faculty of Economics and Business, KU Leuven, Leuven, Belgium

    Wolfgang Glänzel

  2. Amsterdam, The Netherlands

    Henk F. Moed

  3. Competence Center Policy – Industry – Innovation, Fraunhofer Institute for Systems and Innovation Research ISI, Karlsruhe, Germany

    Ulrich Schmoch

  4. Faculty of Science and Engineering, University of Wolverhampton, Wolverhampton, UK

    Mike Thelwall

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer International Publishing AG, part of Springer Nature

About this chapter

Cite this chapter

van Raan, A. (2019). Measuring Science: Basic Principles and Application of Advanced Bibliometrics. In: Glänzel, W., Moed, H.F., Schmoch, U., Thelwall, M. (eds) Springer Handbook of Science and Technology Indicators. Springer Handbooks. Springer, Cham. https://doi.org/10.1007/978-3-030-02511-3_10

Download citation

Publish with us

Policies and ethics

Search

Navigation