Advertisement

Springer Nature is making Coronavirus research free. View research | View latest news | Sign up for updates

Some bibliometric procedures for analyzing and evaluating research fields

Abstract

Nowadays, measuring the quality and quantity of the scientific production is an important necessity since almost every research assessment decision depends, to a great extent, upon the scientific merits of the involved researchers. To do that, many different indicators have been proposed in the literature. Two main bibliometric procedures to explore a research field have been defined: performance analysis and science mapping. On the one hand, performance analysis aims at evaluating groups of scientific actors (countries, universities, departments, researchers) and the impact of their activity on the basis of bibliographic data. On the other hand, the extraction of knowledge from the intellectual, social or conceptual structure of a research field could be done by means of science mapping analysis based on bibliographic networks. In this paper, we introduce some of the most important techniques and software tools to analyze the impact of a research field and its scientific structures. Particularly, four bibliometric indices (h, g, hg and q2), the h-classics approach to identify the classic papers of a research field and three free science mapping software tools (CitNetExplorer, SciMAT and VOSViewer) are shown.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Notes

  1. 1.

    http://www.citnetexplorer.nl/

  2. 2.

    http://sci2s.ugr.es/scimat

  3. 3.

    3 http://www.vosviewver.com

References

  1. 1.

    Alonso S, Cabrerizo FJ, Herrera-Viedma E, Herrera F (2009) h-Index: a review focused in its variants, computation and standardization for different scientific fields. J Informet 3(4):273–289. https://doi.org/10.1016/j.joi.2009.04.001

  2. 2.

    Alonso S, Cabrerizo FJ, Herrera-Viedma E, Herrera F (2010) Hg-index: a new index to characterize the scientific output of researchers based on the h- and g-indices. Scientometrics 82(2):391–400. https://doi.org/10.1007/s11192-009-0047-5

  3. 3.

    Batagelj V, Cerinšek M (2013) On bibliographic networks. Scientometrics 96 (3):845–864. https://doi.org/10.1007/s11192-012-0940-1

  4. 4.

    Batagelj V, Mrvar A (1998) Pajek - program for large network analysis. Connections 21(2):47–58

  5. 5.

    Boccaletti S, Latora V, Moreno Y, Chavez M, Hwang D (2006) Complex networks: structure and dynamics. Phys Rep 424(4–5):175–308. https://doi.org/10.1016/j.physrep.2005.10.009

  6. 6.

    Börner K, Chen C, Boyack KW (2005) Visualizing knowledge domains. Annual Review of Information Science and Technology 37(1):179–255. https://doi.org/10.1002/aris.1440370106

  7. 7.

    Bornmann L, Mutz R, Daniel HD (2008) Are there better indices for evaluation purposes than theh index? a comparison of nine different variants of theh index using data from biomedicine. J Am Soc Inf Sci Technol 59 (5):830–837. https://doi.org/10.1002/asi.20806

  8. 8.

    Burrell QL (2007) On the h-index, the size of the Hirsch core and Jin’s A-index. J Informet 1(2):170–177. https://doi.org/10.1016/j.joi.2007.01.003

  9. 9.

    Cabrerizo F, Alonso S, Herrera-Viedma E, Herrera F (2010) Q2-index: quantitative and qualitative evaluation based on the number and impact of papers in the hirsch core. J Informet 4(1):23–28. https://doi.org/10.1016/j.joi.2009.06.005

  10. 10.

    Callon M, Courtial JP, Turner WA, Bauin S (1983) From translations to problematic networks: an introduction to co-word analysis. https://doi.org/10.1177/053901883022002003

  11. 11.

    Cartes-Velásquez R, Manterola Delgado C (2014) Bibliometric analysis of articles published in ISI dental journals, 2007–2011. Scientometrics 98(3):2223–2233. https://doi.org/10.1007/s11192-013-1173-7

  12. 12.

    Cobo MJ, Chiclana F, Collop A, De Oña j, Herrera-Viedma E (2014) A bibliometric analysis of the intelligent transportation systems research based on science mapping. IEEE Trans Intell Transp Syst 15(2):901–908. https://doi.org/10.1109/TITS.2013.2284756

  13. 13.

    Cobo MJ, López-Herrera AG, Herrera F, Herrera-Viedma E (2012) A note on the ITS topic evolution in the period 2000-2009 at T-ITS. IEEE Trans Intell Transp Syst 13(1):413–420. https://doi.org/10.1109/TITS.2011.2167968

  14. 14.

    Cobo MJ, López-Herrera AG, Herrera-Viedma E, Herrera F (2011) An approach for detecting, quantifying, and visualizing the evolution of a research field: a practical application to the fuzzy sets theory field. J Informet 5(1):146–166. https://doi.org/10.1016/j.joi.2010.10.002

  15. 15.

    Cobo MJ, López-Herrera AG, Herrera-Viedma E, Herrera F (2011) Science mapping software tools: review, analysis, and cooperative study among tools. J Am Soc Inf Sci Technol 62(7):1382–1402. https://doi.org/10.1002/asi.21525

  16. 16.

    Cobo MJ, López-Herrera AG, Herrera-Viedma E, Herrera F (2012) SciMAT: a new science mapping analysis software tool. J Am Soc Inf Sci Technol 63(8):1609–1630. https://doi.org/10.1002/asi.22688

  17. 17.

    Cobo MJ, Martínez MA, Gutiérrez-Salcedo M, Fujita H, Herrera-Viedma E (2015) 25 Years at knowledge-based systems: a bibliometric analysis. Knowl-Based Syst 80:3–13. https://doi.org/10.1016/j.knosys.2014.12.035

  18. 18.

    Cook DJ, Holder LB (2006) Mining graph data. Wiley-Interscience

  19. 19.

    Costas R, Bordons M (2007) The h-index: advantages, limitations and its relation with other bibliometric indicators at the micro level. J Informet 1(3):193–203. https://doi.org/10.1016/j.joi.2007.02.001

  20. 20.

    De la Flor-Martínez M, Galindo-Moreno P, Sánchez-Fernández E, Piattelli A, Cobo MJ, Herrera-Viedma E (2016) H-classic : a new method to identify classic articles in implant dentistry, periodontics, and oral surgery. Clin Oral Implants Res 27(10):1317–1330. https://doi.org/10.1111/clr.12749

  21. 21.

    De Maio C, Fenza G, Loia V, Parente M (2015) Biomedical data integration and ontology-driven multi-facets visualization. In: 2015 international joint conference on neural networks (IJCNN). https://doi.org/10.1109/IJCNN.2015.7280395. IEEE, pp 1–8

  22. 22.

    De Maio C, Parente M, Fenza G, Greco D (2016) Time aware knowledge extraction to analyze nanosafety cluster scientific activities. In: 2016 IEEE congress on evolutionary computation (CEC). IEEE, pp 1233–1240. https://doi.org/10.1109/CEC.2016.7743928

  23. 23.

    Egghe L (2006) Theory and practise of the g-index. Scientometrics 69(1):131–152. https://doi.org/10.1007/s11192-006-0144-7

  24. 24.

    Egghe L, Rousseau R (2008) An h-index weighted by citation impact. Inf Process Manag 44(2):770–780. https://doi.org/10.1016/j.ipm.2007.05.003

  25. 25.

    Feijoo JF, Limeres J, Fernández-Varela M, Ramos I, Diz P (2014) The 100 most cited articles in dentistry. Clinical Oral Investigations 18(3):699–706. https://doi.org/10.1007/s00784-013-1017-0

  26. 26.

    Garfield E (1972) Citation analysis as a tool in journal evaluation. Science 178(4060):471–479. https://doi.org/10.1126/science.178.4060.471

  27. 27.

    Garfield E (1977) Introducing citation classics. The human side of scientific reports. Current Comments 1 (1):5–7

  28. 28.

    Garfield E (1979) Citation indexing: its theory and application in science, technology, and humanities. Wiley, New York

  29. 29.

    Garfield E (1987) 100 citation classics from the journal of the american medical association. J Am Med Assoc 257(1):52. https://doi.org/10.1001/jama.1987.03390010056028

  30. 30.

    Garfield E (1994) Scientography: mapping the tracks of science. Current Contents: Social & Behavioural Sciences 7(45):5–10

  31. 31.

    Garfield E, Pudovkin AI, Istomin VS (2003) Why do we need algorithmic historiography? J Am Soc Inf Sci Technol 54(5):400–412. https://doi.org/10.1002/asi.10226

  32. 32.

    Glanzel W (2001) National characteristics in international scientific co-authorship relations. Scientometrics 51(1):69–115. https://doi.org/10.1023/A:1010512628145

  33. 33.

    Hirsch JE (2005) An index to quantify an individual’s scientific research output. Proc Natl Acad Sci 102 (46):16,569–16,572. https://doi.org/10.1073/pnas.0507655102

  34. 34.

    Houari NS, Taghezout N (2016) Integrating agents into a collaborative knowledge-based system for business rules consistency management. International Journal of Interactive Multimedia and Artificial Intelligence 4(2):61–72. https://doi.org/10.9781/ijimai.2016.4210

  35. 35.

    Huang MH, Chang CP (2014) Detecting research fronts in OLED field using bibliographic coupling with sliding window. Scientometrics 98(3):1721–1744. https://doi.org/10.1007/s11192-013-1126-1

  36. 36.

    Hutchins BI, Yuan X, Anderson JM, Santangelo GM (2016) Relative citation ratio (RCR): a new metric that uses citation rates to measure influence at the article level. PLOS Biology 14(9):e1002,541. https://doi.org/10.1371/journal.pbio.1002541

  37. 37.

    Ibrahim GM, Carter Snead O, Rutka JT, Lozano AM (2012) The most cited works in epilepsy: trends in the “Citation Classics”. Epilepsia 53(5):765–770. https://doi.org/10.1111/j.1528-1167.2012.03455.x

  38. 38.

    Jin B (2006) h-Index: an evaluation indicator proposed by scientist. Science Focus 1(1):8–9

  39. 39.

    Jin B, Liang L, Rousseau R, Egghe L (2007) The R- and AR-indices: complementing the h-index. Chin Sci Bull 52(6):855–863. https://doi.org/10.1007/s11434-007-0145-9

  40. 40.

    Kessler MM (1963) Bibliographic coupling between scientific papers. Am Doc 14(1):10–25. https://doi.org/10.1002/asi.5090140103

  41. 41.

    Li X, Zhou Y, Xue L, Huang L (2015) Integrating bibliometrics and roadmapping methods: a case of dye-sensitized solar cell technology-based industry in China. Technol Forecast Soc Chang 97:205–222. https://doi.org/10.1016/j.techfore.2014.05.007

  42. 42.

    Martínez MA, Cobo MJ, Herrera M, Herrera-Viedma E (2015) Analyzing the scientific evolution of social work using science mapping. Res Soc Work Pract 5(2):257–277. https://doi.org/10.1177/1049731514522101

  43. 43.

    Martínez MA, Herrera M, López-Gijón J, Herrera-Viedma E (2014) H-classics: characterizing the concept of citation classics through H-index. Scientometrics 98(3):1971–1983. https://doi.org/10.1007/s11192-013-1155-9

  44. 44.

    Moed HF, Bruin RE, Leeuwen TN (1995) New bibliometric tools for the assessment of national research performance: Database description, overview of indicators and first applications. Scientometrics 33(3):381–422. https://doi.org/10.1007/BF02017338

  45. 45.

    Moral-Muñoz JA, Cobo MJ, Chiclana F, Collop A, Herrera-Viedma E (2016) Analyzing highly cited papers in intelligent transportation systems. IEEE Trans Intell Transp Syst 17(4):993–1001. https://doi.org/10.1109/TITS.2015.2494533

  46. 46.

    Moral-Muñoz JA, Cobo MJ, Peis E, Arroyo-Morales M, Herrera-Viedma E (2014) Analyzing the research in integrative complementary medicine by means of science mapping. Complement Ther Med 22(2):409–418. https://doi.org/10.1016/j.ctim.2014.02.003

  47. 47.

    Murgado-Armenteros EM, Gutiérrez-Salcedo M, Torres-Ruiz FJ, Cobo MJ (2015) Analysing the conceptual evolution of qualitative marketing research through science mapping analysis. Scientometrics 102 (1):519–557. https://doi.org/10.1007/s11192-014-1443-z

  48. 48.

    Noyons ECM, Moed HF, Luwel M (1999) Combining mapping and citation analysis for evaluative bibliometric purposes: A bibliometric study. J Am Soc Inf Sci 50(2):115–131. https://doi.org/10.1002/(SICI)1097-4571(1999)50:2⟨115::AID-ASI3⟩3.0.CO;2-J

  49. 49.

    Perianes-Rodriguez A, Waltman L, van Eck NJ (2016) Constructing bibliometric networks: a comparison between full and fractional counting. J Informet 10(4):1178–1195. https://doi.org/10.1016/j.joi.2016.10.006

  50. 50.

    Ponce FA, Lozano AM (2011) The most cited works in Parkinson’s disease. Mov Disord 26(3):380–390. https://doi.org/10.1002/mds.23445

  51. 51.

    Rodriguez-Ledesma A, Cobo MJ, Lopez-Pujalte C, Herrera-Viedma E (2015) An overview of animal science research 1945-2011 through science mapping analysis. J Anim Breeding Genet 132(6):475–497. https://doi.org/10.1111/jbg.12124

  52. 52.

    Rousseau R (2006) New developments related to the Hirsch index. Science Focus 1(4):23–25

  53. 53.

    SCImago (2007) SJR - SCIMago Journal & Country Rank

  54. 54.

    Settouti N, Bechar MEA, Chikh MA (2016) Statistical comparisons of the top 10 algorithms in data mining for classi cation task. International Journal of Interactive Multimedia and Artificial Intelligence 4(1):46–51. https://doi.org/10.9781/ijimai.2016.419

  55. 55.

    Sidiropoulos A, Katsaros D, Manolopoulos Y (2007) Generalized Hirsch h-index for disclosing latent facts in citation networks. Scientometrics 72(2):253–280. https://doi.org/10.1007/s11192-007-1722-z

  56. 56.

    Small H (1973) Co-citation in the scientific literature: a new measure of the relationship between two documents. J Am Soc Inf Sci 24(4):265–269. https://doi.org/10.1002/asi.4630240406

  57. 57.

    Small H (1999) Visualizing science by citation mapping. J Am Soc Inf Sci 50 (9):799–813. https://doi.org/10.1002/(SICI)1097-4571(1999)50:9⟨799::AID-ASI9⟩3.0.CO;2-G

  58. 58.

    Smith D (2007) Ten citation classics from the New Zealand medical journal. N Z Med J 120(1267):2871–2875

  59. 59.

    Stack S (2013) Citation classics in deviant behavior: a research note. Deviant Behavior 34(2):85–96. https://doi.org/10.1080/01639625.2012.707539

  60. 60.

    Tam WW, Wong EL, Wong FC, Cheung AW (2012) Citation classics in the integrative and complementary medicine literature: 50 frequently cited articles. Eur J Intern Med 4(1):e77–e83. https://doi.org/10.1016/j.eujim.2011.12.004

  61. 61.

    Tang L, Shapira P (2011) China-US scientific collaboration in nanotechnology: patterns and dynamics. Scientometrics 88(1):1–16. https://doi.org/10.1007/s11192-011-0376-z

  62. 62.

    Van Eck NJ, Waltman L (2009) How to normalize cooccurrence data? An analysis of some well-known similarity measures. J Am Soc Inf Sci Technol 60(8):1635–1651. https://doi.org/10.1002/asi.21075

  63. 63.

    van Eck NJ, Waltman L (2010) Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 84(2):523–538. https://doi.org/10.1007/s11192-009-0146-3

  64. 64.

    van Eck NJ, Waltman L (2014) Citnetexplorer: a new software tool for analyzing and visualizing citation networks. J Informet 8(4):802–823. https://doi.org/10.1016/j.joi.2014.07.006

  65. 65.

    van Raan AFJ (2005) Measuring science. In: Moed HF, Glänzel W, Schmoch U (eds) Handbook of quantitative science and technology research, chap. measuring. Springer, Netherlands, pp 19–50

  66. 66.

    Vanclay JK (2007) On the robustness of theh-index. J Am Soc Inf Sci Technol 58(10):1547–1550. https://doi.org/10.1002/asi.20616

Download references

Acknowledgements

The authors would like to acknowledge FEDER funds under grants TIN2013-40658-P and TIN2016-75850-R, and also the financial support from the University of Cádiz Project PR2016-067.

Author information

Correspondence to E. Herrera-Viedma.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gutiérrez-Salcedo, M., Martínez, M.Á., Moral-Munoz, J.A. et al. Some bibliometric procedures for analyzing and evaluating research fields. Appl Intell 48, 1275–1287 (2018). https://doi.org/10.1007/s10489-017-1105-y

Download citation

Keywords

  • Bibliometrics
  • H-index
  • Science mapping
  • Citations