, Volume 79, Issue 3, pp 473–490 | Cite as

Expansion of scientific journal categories using reference analysis: How can it be done and does it make a difference?

  • Carmen López-Illescas
  • Ed C.M. Noyons
  • Martijn S. Visser
  • Félix De Moya-Anegón
  • Henk F. Moed


This paper explores a methodology for delimitating scientific subfields by combining the use of (specialist) journal categories from Thomson Scientific’s Web of Science (WoS) and reference analysis. In a first step it selects all articles in journals included in a particular WoS journal category covering a subfield. These journals are labelled as a subfield’s specialist journals. In a second step, this set of papers is expanded with papers published in other, additional journals and citing a subfield’s specialist journals with a frequency exceeding a certain citation threshold. Data are presented for two medical subfields: Oncology and Cardiac & Cardiovascular System. A validation based on findings from earlier studies, from an analysis of MESH descriptors from MEDLINE, and on expert opinion provides evidence that the proposed methodology has a high precision, and that expansion substantially enhanced the recall, not merely in terms of the number of retrieved papers, but also in terms of the number of research topics covered. The paper also examines how a bibliometric ranking of countries and universities based on the citation impact of their papers published in a subfield’s specialist journals compares to that of a ranking based on the impact of their articles in additional journals. Rather weak correlations especially obtained at the level of universities underline the conclusion from earlier studies that an assessment of research groups or universities in a scientific subfield that takes into account solely papers published in a subfield’s specialist journals is unsatisfactory.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aksnes, D. W., Olsen, T. B., Seglen, P. O. (2000), Validation of bibliometric indicators in the field of microbiology: A Norwegian case study. Scientometrics, 49 : 7–22.CrossRefGoogle Scholar
  2. Cambrosio, A., Keating, P., Mercier, S., Lewison, G., Mogoutov, A. (2006), Mapping the emergence and development of translational cancer research. Eur J Cancer, 42: 3140–3148.CrossRefGoogle Scholar
  3. Garfield, E. (1964), The Citation Index - A new dimension in indexing. Science, 144: 649–654.CrossRefGoogle Scholar
  4. Glänzel, W., Schubert, A., Schoepflin, U., Czerwon, H. J. (1999A), An item-by-item subject classification of papers published in multidisciplinary and general journals using reference analysis. Scientometrics, 44: 427–439.CrossRefGoogle Scholar
  5. Glänzel, W., Schubert, A., Schoepflin, U., Czerwon, H. J. (1999B), An item-by-item subject classification of papers published in journals covered by the SSCI database using reference analysis. Scientometrics, 46: 431–441.CrossRefGoogle Scholar
  6. Lewison, G. (1996), The definition of biomedical research subfields with title keywords and application to the analysis of research outputs. Research Evaluation, 6: 25–36.Google Scholar
  7. Lewison, G. (1999), The definition and calibration of biomedical subfields. Scientometrics, 46: 529–537.CrossRefGoogle Scholar
  8. Lewison, G. (2003), The publication of cancer research papers in high impact journals. ASLIB Proc., 55: 379–387.CrossRefGoogle Scholar
  9. López-Illescas, C., Moya-Anegón, F., Moed, H. F. (2008), The actual citation impact of European oncological research. European Journal of Cancer, to be published.Google Scholar
  10. Moed, H. F., De Bruin, R. E., Van Leeuwen, T. N. (1995), New bibliometric tools for the assessment of national research performance: database description, overview of indicators and first applications. Scientometrics, 33: 381–442.CrossRefGoogle Scholar
  11. Moed, H. F. (2005), Citation Analysis in Research Evaluation. Dordrecht (the Netherlands): Springer, 346 pp.Google Scholar
  12. Moed, H. F. (2006), Bibliometric Rankings of World Universities. CWTS Report 2006-01. Available at
  13. Moya-Anegon, F., Herrero-Solana, V., Jimenez-Contreras, E. (2006), A connectionist and multivariate approach to science maps: the SOM, clustering and MDS applied to library science research and information. Journal of Information Science, 32: 63–77.CrossRefGoogle Scholar
  14. Noyons, E. C. M. (2004), Science maps within a science policy context. In: Moed, H. F., Glänzel, W., Schmoch, U. (Eds), Handbook of Quantitative Science and Technology Research. The Use of Publication and Patent Statistics in Studies of S&T Systems. Dordrecht (the Netherlands): Kluwer Academic Publishers, 237–256.Google Scholar
  15. Pinski, G., Narin, F. (1976), Citation influence for journal aggregates of scientific publications: theory, with application to the literature of physics. Information Processing and Management, 12: 297–312.CrossRefGoogle Scholar
  16. Small, H. (1973), Co-citation in the scientific literature: A new measure of the relationship between publications. Journal of the American Society for Information Science, 24: 265–269.CrossRefGoogle Scholar
  17. Small, H. (1987), The significance of bibliographic references. Scientometrics, 12: 339–342.CrossRefGoogle Scholar
  18. Small, H. (1998), A general framework for creating large-scale maps of science in two or three dimensions: The SciViz system. Scientometrics, 41: 125–133.CrossRefGoogle Scholar
  19. Ugolini, D., Casilli, C., Mela, G. S. (2002), Assesing oncological productivity: is one method sufficient? Eur J Cancer, 38: 1121–1125.CrossRefGoogle Scholar
  20. Ugolini, D., Mela, G. S. (2003), Oncological research overview in the European Union. A 5-year survey. Eur J Cancer, 39: 1888–1894.CrossRefGoogle Scholar
  21. Van Leeuwen, T. N., Van Der Wurff, L. J., Van Raan, A. F. J. (2001), The use of combined bibliometric methods in research funding policy. Research Evaluation 10: 195–201.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2009

Authors and Affiliations

  • Carmen López-Illescas
    • 1
  • Ed C.M. Noyons
    • 2
  • Martijn S. Visser
    • 2
  • Félix De Moya-Anegón
    • 3
  • Henk F. Moed
    • 2
  1. 1.Scimago Group, Department of Library and Information ScienceUniversity of GranadaGranadaSpain
  2. 2.Centre for Science and Technology Studies (CWTS)Leiden UniversityLeidenThe Netherlands
  3. 3.Scimago GroupSpanish National Research Council (CSIC)MadridSpain

Personalised recommendations