Applied Intelligence

, Volume 48, Issue 5, pp 1275–1287 | Cite as

Some bibliometric procedures for analyzing and evaluating research fields

  • M. Gutiérrez-Salcedo
  • M. Ángeles Martínez
  • J. A. Moral-Munoz
  • E. Herrera-Viedma
  • M. J. Cobo


Nowadays, measuring the quality and quantity of the scientific production is an important necessity since almost every research assessment decision depends, to a great extent, upon the scientific merits of the involved researchers. To do that, many different indicators have been proposed in the literature. Two main bibliometric procedures to explore a research field have been defined: performance analysis and science mapping. On the one hand, performance analysis aims at evaluating groups of scientific actors (countries, universities, departments, researchers) and the impact of their activity on the basis of bibliographic data. On the other hand, the extraction of knowledge from the intellectual, social or conceptual structure of a research field could be done by means of science mapping analysis based on bibliographic networks. In this paper, we introduce some of the most important techniques and software tools to analyze the impact of a research field and its scientific structures. Particularly, four bibliometric indices (h, g, hg and q2), the h-classics approach to identify the classic papers of a research field and three free science mapping software tools (CitNetExplorer, SciMAT and VOSViewer) are shown.


Bibliometrics H-index Science mapping Citations 



The authors would like to acknowledge FEDER funds under grants TIN2013-40658-P and TIN2016-75850-R, and also the financial support from the University of Cádiz Project PR2016-067.


  1. 1.
    Alonso S, Cabrerizo FJ, Herrera-Viedma E, Herrera F (2009) h-Index: a review focused in its variants, computation and standardization for different scientific fields. J Informet 3(4):273–289. CrossRefGoogle Scholar
  2. 2.
    Alonso S, Cabrerizo FJ, Herrera-Viedma E, Herrera F (2010) Hg-index: a new index to characterize the scientific output of researchers based on the h- and g-indices. Scientometrics 82(2):391–400. CrossRefGoogle Scholar
  3. 3.
    Batagelj V, Cerinšek M (2013) On bibliographic networks. Scientometrics 96 (3):845–864. CrossRefGoogle Scholar
  4. 4.
    Batagelj V, Mrvar A (1998) Pajek - program for large network analysis. Connections 21(2):47–58zbMATHGoogle Scholar
  5. 5.
    Boccaletti S, Latora V, Moreno Y, Chavez M, Hwang D (2006) Complex networks: structure and dynamics. Phys Rep 424(4–5):175–308. MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Börner K, Chen C, Boyack KW (2005) Visualizing knowledge domains. Annual Review of Information Science and Technology 37(1):179–255. CrossRefGoogle Scholar
  7. 7.
    Bornmann L, Mutz R, Daniel HD (2008) Are there better indices for evaluation purposes than theh index? a comparison of nine different variants of theh index using data from biomedicine. J Am Soc Inf Sci Technol 59 (5):830–837. CrossRefGoogle Scholar
  8. 8.
    Burrell QL (2007) On the h-index, the size of the Hirsch core and Jin’s A-index. J Informet 1(2):170–177. MathSciNetCrossRefGoogle Scholar
  9. 9.
    Cabrerizo F, Alonso S, Herrera-Viedma E, Herrera F (2010) Q2-index: quantitative and qualitative evaluation based on the number and impact of papers in the hirsch core. J Informet 4(1):23–28. CrossRefGoogle Scholar
  10. 10.
    Callon M, Courtial JP, Turner WA, Bauin S (1983) From translations to problematic networks: an introduction to co-word analysis.
  11. 11.
    Cartes-Velásquez R, Manterola Delgado C (2014) Bibliometric analysis of articles published in ISI dental journals, 2007–2011. Scientometrics 98(3):2223–2233. CrossRefGoogle Scholar
  12. 12.
    Cobo MJ, Chiclana F, Collop A, De Oña j, Herrera-Viedma E (2014) A bibliometric analysis of the intelligent transportation systems research based on science mapping. IEEE Trans Intell Transp Syst 15(2):901–908. CrossRefGoogle Scholar
  13. 13.
    Cobo MJ, López-Herrera AG, Herrera F, Herrera-Viedma E (2012) A note on the ITS topic evolution in the period 2000-2009 at T-ITS. IEEE Trans Intell Transp Syst 13(1):413–420. CrossRefzbMATHGoogle Scholar
  14. 14.
    Cobo MJ, López-Herrera AG, Herrera-Viedma E, Herrera F (2011) An approach for detecting, quantifying, and visualizing the evolution of a research field: a practical application to the fuzzy sets theory field. J Informet 5(1):146–166. CrossRefzbMATHGoogle Scholar
  15. 15.
    Cobo MJ, López-Herrera AG, Herrera-Viedma E, Herrera F (2011) Science mapping software tools: review, analysis, and cooperative study among tools. J Am Soc Inf Sci Technol 62(7):1382–1402. CrossRefzbMATHGoogle Scholar
  16. 16.
    Cobo MJ, López-Herrera AG, Herrera-Viedma E, Herrera F (2012) SciMAT: a new science mapping analysis software tool. J Am Soc Inf Sci Technol 63(8):1609–1630. CrossRefzbMATHGoogle Scholar
  17. 17.
    Cobo MJ, Martínez MA, Gutiérrez-Salcedo M, Fujita H, Herrera-Viedma E (2015) 25 Years at knowledge-based systems: a bibliometric analysis. Knowl-Based Syst 80:3–13. CrossRefGoogle Scholar
  18. 18.
    Cook DJ, Holder LB (2006) Mining graph data. Wiley-InterscienceGoogle Scholar
  19. 19.
    Costas R, Bordons M (2007) The h-index: advantages, limitations and its relation with other bibliometric indicators at the micro level. J Informet 1(3):193–203. CrossRefGoogle Scholar
  20. 20.
    De la Flor-Martínez M, Galindo-Moreno P, Sánchez-Fernández E, Piattelli A, Cobo MJ, Herrera-Viedma E (2016) H-classic : a new method to identify classic articles in implant dentistry, periodontics, and oral surgery. Clin Oral Implants Res 27(10):1317–1330. CrossRefGoogle Scholar
  21. 21.
    De Maio C, Fenza G, Loia V, Parente M (2015) Biomedical data integration and ontology-driven multi-facets visualization. In: 2015 international joint conference on neural networks (IJCNN). IEEE, pp 1–8
  22. 22.
    De Maio C, Parente M, Fenza G, Greco D (2016) Time aware knowledge extraction to analyze nanosafety cluster scientific activities. In: 2016 IEEE congress on evolutionary computation (CEC). IEEE, pp 1233–1240.
  23. 23.
    Egghe L (2006) Theory and practise of the g-index. Scientometrics 69(1):131–152. MathSciNetCrossRefGoogle Scholar
  24. 24.
    Egghe L, Rousseau R (2008) An h-index weighted by citation impact. Inf Process Manag 44(2):770–780. CrossRefGoogle Scholar
  25. 25.
    Feijoo JF, Limeres J, Fernández-Varela M, Ramos I, Diz P (2014) The 100 most cited articles in dentistry. Clinical Oral Investigations 18(3):699–706. CrossRefGoogle Scholar
  26. 26.
    Garfield E (1972) Citation analysis as a tool in journal evaluation. Science 178(4060):471–479. CrossRefGoogle Scholar
  27. 27.
    Garfield E (1977) Introducing citation classics. The human side of scientific reports. Current Comments 1 (1):5–7Google Scholar
  28. 28.
    Garfield E (1979) Citation indexing: its theory and application in science, technology, and humanities. Wiley, New YorkGoogle Scholar
  29. 29.
    Garfield E (1987) 100 citation classics from the journal of the american medical association. J Am Med Assoc 257(1):52. CrossRefGoogle Scholar
  30. 30.
    Garfield E (1994) Scientography: mapping the tracks of science. Current Contents: Social & Behavioural Sciences 7(45):5–10Google Scholar
  31. 31.
    Garfield E, Pudovkin AI, Istomin VS (2003) Why do we need algorithmic historiography? J Am Soc Inf Sci Technol 54(5):400–412. CrossRefGoogle Scholar
  32. 32.
    Glanzel W (2001) National characteristics in international scientific co-authorship relations. Scientometrics 51(1):69–115. MathSciNetCrossRefGoogle Scholar
  33. 33.
    Hirsch JE (2005) An index to quantify an individual’s scientific research output. Proc Natl Acad Sci 102 (46):16,569–16,572. CrossRefzbMATHGoogle Scholar
  34. 34.
    Houari NS, Taghezout N (2016) Integrating agents into a collaborative knowledge-based system for business rules consistency management. International Journal of Interactive Multimedia and Artificial Intelligence 4(2):61–72. CrossRefGoogle Scholar
  35. 35.
    Huang MH, Chang CP (2014) Detecting research fronts in OLED field using bibliographic coupling with sliding window. Scientometrics 98(3):1721–1744. CrossRefGoogle Scholar
  36. 36.
    Hutchins BI, Yuan X, Anderson JM, Santangelo GM (2016) Relative citation ratio (RCR): a new metric that uses citation rates to measure influence at the article level. PLOS Biology 14(9):e1002,541. CrossRefGoogle Scholar
  37. 37.
    Ibrahim GM, Carter Snead O, Rutka JT, Lozano AM (2012) The most cited works in epilepsy: trends in the “Citation Classics”. Epilepsia 53(5):765–770. CrossRefGoogle Scholar
  38. 38.
    Jin B (2006) h-Index: an evaluation indicator proposed by scientist. Science Focus 1(1):8–9MathSciNetGoogle Scholar
  39. 39.
    Jin B, Liang L, Rousseau R, Egghe L (2007) The R- and AR-indices: complementing the h-index. Chin Sci Bull 52(6):855–863. CrossRefGoogle Scholar
  40. 40.
    Kessler MM (1963) Bibliographic coupling between scientific papers. Am Doc 14(1):10–25. CrossRefGoogle Scholar
  41. 41.
    Li X, Zhou Y, Xue L, Huang L (2015) Integrating bibliometrics and roadmapping methods: a case of dye-sensitized solar cell technology-based industry in China. Technol Forecast Soc Chang 97:205–222. CrossRefGoogle Scholar
  42. 42.
    Martínez MA, Cobo MJ, Herrera M, Herrera-Viedma E (2015) Analyzing the scientific evolution of social work using science mapping. Res Soc Work Pract 5(2):257–277. CrossRefGoogle Scholar
  43. 43.
    Martínez MA, Herrera M, López-Gijón J, Herrera-Viedma E (2014) H-classics: characterizing the concept of citation classics through H-index. Scientometrics 98(3):1971–1983. CrossRefGoogle Scholar
  44. 44.
    Moed HF, Bruin RE, Leeuwen TN (1995) New bibliometric tools for the assessment of national research performance: Database description, overview of indicators and first applications. Scientometrics 33(3):381–422. CrossRefGoogle Scholar
  45. 45.
    Moral-Muñoz JA, Cobo MJ, Chiclana F, Collop A, Herrera-Viedma E (2016) Analyzing highly cited papers in intelligent transportation systems. IEEE Trans Intell Transp Syst 17(4):993–1001. CrossRefGoogle Scholar
  46. 46.
    Moral-Muñoz JA, Cobo MJ, Peis E, Arroyo-Morales M, Herrera-Viedma E (2014) Analyzing the research in integrative complementary medicine by means of science mapping. Complement Ther Med 22(2):409–418. CrossRefGoogle Scholar
  47. 47.
    Murgado-Armenteros EM, Gutiérrez-Salcedo M, Torres-Ruiz FJ, Cobo MJ (2015) Analysing the conceptual evolution of qualitative marketing research through science mapping analysis. Scientometrics 102 (1):519–557. CrossRefGoogle Scholar
  48. 48.
    Noyons ECM, Moed HF, Luwel M (1999) Combining mapping and citation analysis for evaluative bibliometric purposes: A bibliometric study. J Am Soc Inf Sci 50(2):115–131.⟨115::AID-ASI3⟩3.0.CO;2-J CrossRefGoogle Scholar
  49. 49.
    Perianes-Rodriguez A, Waltman L, van Eck NJ (2016) Constructing bibliometric networks: a comparison between full and fractional counting. J Informet 10(4):1178–1195. CrossRefGoogle Scholar
  50. 50.
    Ponce FA, Lozano AM (2011) The most cited works in Parkinson’s disease. Mov Disord 26(3):380–390. CrossRefGoogle Scholar
  51. 51.
    Rodriguez-Ledesma A, Cobo MJ, Lopez-Pujalte C, Herrera-Viedma E (2015) An overview of animal science research 1945-2011 through science mapping analysis. J Anim Breeding Genet 132(6):475–497. CrossRefGoogle Scholar
  52. 52.
    Rousseau R (2006) New developments related to the Hirsch index. Science Focus 1(4):23–25Google Scholar
  53. 53.
    SCImago (2007) SJR - SCIMago Journal & Country RankGoogle Scholar
  54. 54.
    Settouti N, Bechar MEA, Chikh MA (2016) Statistical comparisons of the top 10 algorithms in data mining for classi cation task. International Journal of Interactive Multimedia and Artificial Intelligence 4(1):46–51. CrossRefGoogle Scholar
  55. 55.
    Sidiropoulos A, Katsaros D, Manolopoulos Y (2007) Generalized Hirsch h-index for disclosing latent facts in citation networks. Scientometrics 72(2):253–280. CrossRefGoogle Scholar
  56. 56.
    Small H (1973) Co-citation in the scientific literature: a new measure of the relationship between two documents. J Am Soc Inf Sci 24(4):265–269. CrossRefGoogle Scholar
  57. 57.
    Small H (1999) Visualizing science by citation mapping. J Am Soc Inf Sci 50 (9):799–813.⟨799::AID-ASI9⟩3.0.CO;2-G CrossRefGoogle Scholar
  58. 58.
    Smith D (2007) Ten citation classics from the New Zealand medical journal. N Z Med J 120(1267):2871–2875Google Scholar
  59. 59.
    Stack S (2013) Citation classics in deviant behavior: a research note. Deviant Behavior 34(2):85–96. CrossRefGoogle Scholar
  60. 60.
    Tam WW, Wong EL, Wong FC, Cheung AW (2012) Citation classics in the integrative and complementary medicine literature: 50 frequently cited articles. Eur J Intern Med 4(1):e77–e83. CrossRefGoogle Scholar
  61. 61.
    Tang L, Shapira P (2011) China-US scientific collaboration in nanotechnology: patterns and dynamics. Scientometrics 88(1):1–16. CrossRefGoogle Scholar
  62. 62.
    Van Eck NJ, Waltman L (2009) How to normalize cooccurrence data? An analysis of some well-known similarity measures. J Am Soc Inf Sci Technol 60(8):1635–1651. CrossRefGoogle Scholar
  63. 63.
    van Eck NJ, Waltman L (2010) Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 84(2):523–538. CrossRefGoogle Scholar
  64. 64.
    van Eck NJ, Waltman L (2014) Citnetexplorer: a new software tool for analyzing and visualizing citation networks. J Informet 8(4):802–823. CrossRefGoogle Scholar
  65. 65.
    van Raan AFJ (2005) Measuring science. In: Moed HF, Glänzel W, Schmoch U (eds) Handbook of quantitative science and technology research, chap. measuring. Springer, Netherlands, pp 19–50Google Scholar
  66. 66.
    Vanclay JK (2007) On the robustness of theh-index. J Am Soc Inf Sci Technol 58(10):1547–1550. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2017

Authors and Affiliations

  • M. Gutiérrez-Salcedo
    • 1
  • M. Ángeles Martínez
    • 2
  • J. A. Moral-Munoz
    • 3
    • 4
  • E. Herrera-Viedma
    • 5
  • M. J. Cobo
    • 6
  1. 1.Department of Management and MarketingUniversity of JaénJaénSpain
  2. 2.Department of Social Work and Social ServicesUniversity of GranadaGranadaSpain
  3. 3.Department of Nursing and PhysiotherapyUniversity of CádizCádizSpain
  4. 4.Institute of Research and Innovation in Biomedical Sciences of the Province of Cadiz (INiBICA)University of CádizCádizSpain
  5. 5.Department of Computer Science and A.I.University of GranadaGranadaSpain
  6. 6.Department of Computer Science and EngineeringUniversity of CádizAlgecirasSpain

Personalised recommendations