Advertisement

Current concepts on bibliometrics: a brief review about impact factor, Eigenfactor score, CiteScore, SCImago Journal Rank, Source-Normalised Impact per Paper, H-index, and alternative metrics

  • Ernesto Roldan-Valadez
  • Shirley Yoselin Salazar-Ruiz
  • Rafael Ibarra-Contreras
  • Camilo Rios
Review Article
  • 50 Downloads

Abstract

Background

Understanding the impact of a publication by using bibliometric indices becomes an essential activity not only for universities and research institutes but also for individual academicians. This paper aims to provide a brief review of the current bibliometric tools used by authors and editors and proposes an algorithm to assess the relevance of the most common bibliometric tools to help the researchers select the fittest journal and know the trends of published submissions by using self-evaluation.

Methods

We present a narrative review answering at least two related consecutive questions triggered by the topics mentioned above. How prestigious is a journal based on its most recent bibliometrics, so authors may choose it to submit their next manuscript? And, how can they self-evaluate/understand the impact of their whole publishing scientific life?

Results

We presented the main relevant definitions of each bibliometrics and grouped them in those oriented to evaluated journals or individuals. Also, we share with our readers our algorithm to assess journals before manuscript submission.

Conclusions

Since there is a journal performance market and an article performance market, each one with its patterns, an integrative use of these metrics, rather than just the impact factor alone, might represent the fairest and most legitimate approach to assess the influence and importance of an acceptable research issue, and not only a sound journal in their respective disciplines.

Keywords

Algorithms Bibliometrics Citation Journal impact factor Self-evaluation 

Notes

Acknowledgements

Shirley Yoselin Salazar-Ruiz, M.D., was a research fellow at Directorate of Research, Hospital General de Mexico “Dr Eduardo Liceaga”, in 2018–2019.

No IRB approval was required for this manuscript.

Compliance with ethical standards

This article does not contain any studies with human participants or animals performed by any of the authors.

For this type of formal study, consent is not required.

Conflict of interest

The authors declare that they have no conflict of interest.

References

  1. 1.
    Jones T, Huggett S, Kamalski J (2011) Finding a way through the scientific literature: indexes and measures. World neurosurgery 76(1–2):36–38.  https://doi.org/10.1016/j.wneu.2011.01.015 CrossRefPubMedGoogle Scholar
  2. 2.
    Eyre-Walker A, Stoletzki N (2013) The assessment of science: the relative merits of post-publication review, the impact factor, and the number of citations. PLoS Biol 11(10):e1001675.  https://doi.org/10.1371/journal.pbio.1001675 CrossRefPubMedPubMedCentralGoogle Scholar
  3. 3.
    Weinstein JN (2007) Threats to scientific advancement in clinical practice. Spine 32(11 Suppl):S58–S62.  https://doi.org/10.1097/BRS.0b013e318053d4fc CrossRefPubMedGoogle Scholar
  4. 4.
    Gasparyan AY (2010) Thoughts on impact factors and editing of medical journals. Inflamm Allergy Drug Targets 9(1):2–5CrossRefPubMedGoogle Scholar
  5. 5.
    Monastersky R (2005) The number that is devouring science. Chron High EducGoogle Scholar
  6. 6.
    de Solla Price DJ (1965) Networks of scientific papers. Science 149(3683):510–515.  https://doi.org/10.1126/science.149.3683.510 CrossRefGoogle Scholar
  7. 7.
    Bergstrom C (2007) Eigenfactor: measuring the value and prestige of scholarly journals. Coll Res Libr News 68(5):314–316CrossRefGoogle Scholar
  8. 8.
    Davis PM (2002) Where to spend our e-journal money? In: press oHU (ed) defining a university library's core collection through citation analysis, vol 2. Baltimore, USA, pp 155–166Google Scholar
  9. 9.
    Garfield E (1996) How can impact factors be improved? BMJ 313(7054):411–413CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Schoenbach UH, Garfield E (1956) Citation indexes for science. Science 123(3185):61–62CrossRefPubMedGoogle Scholar
  11. 11.
    Adam D (2002) The counting house. Nature 415(6873):726–729.  https://doi.org/10.1038/415726a CrossRefPubMedGoogle Scholar
  12. 12.
    Parrillo JE (2005) Our journal, critical care medicine, in 2005: high impact factor, rapid manuscript review, growing submissions, and widespread distribution. Crit Care Med 33(5):923–924CrossRefPubMedGoogle Scholar
  13. 13.
    Wikipedia (2011) Impact factor. WikipediaGoogle Scholar
  14. 14.
    Mathur VP, Sharma A (2009) Impact factor and other standardized measures of journal citation: a perspective. Indian J Dent Res 20(1):81–85CrossRefPubMedGoogle Scholar
  15. 15.
    Thomson_Reuters (2014) Web of Knowledge. http://wokinfo.com. Accessed Accesed 19 May 2014
  16. 16.
    Kumar V, Upadhyay S, Medhi B (2009) Impact of the impact factor in biomedical research: its use and misuse. Singap Med J 50(8):752–755Google Scholar
  17. 17.
    Horgan A (2002) BMJ's impact factor increases by 24. BMJ 325(7354):8CrossRefPubMedPubMedCentralGoogle Scholar
  18. 18.
    Tobin MJ (2004) Thirty years of impact factor and the journal. Am J Respir Crit Care Med 170(4):351–352.  https://doi.org/10.1164/rccm.2406005 CrossRefPubMedGoogle Scholar
  19. 19.
    Wedzicha JA, Johnston SL, Mitchell DM (2005) Journal impact factors for 2004: another rise for thorax. Thorax 60(9):712.  https://doi.org/10.1136/thx.2005.050922 CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Rizkallah J, Sin DD (2010) Integrative approach to quality assessment of medical journals using impact factor, eigenfactor, and article influence scores. PLoS One 5(4):e10204.  https://doi.org/10.1371/journal.pone.0010204 CrossRefPubMedPubMedCentralGoogle Scholar
  21. 21.
    Thomaz PG, Assad RS, Moreira LF (2011) Using the impact factor and H index to assess researchers and publications. Arq Bras Cardiol 96(2):90–93CrossRefPubMedGoogle Scholar
  22. 22.
    Saha S, Saint S, Christakis DA (2003) Impact factor: a valid measure of journal quality? Journal of the Medical Library Association : JMLA 91(1):42–46PubMedGoogle Scholar
  23. 23.
    Bergstrom CT, West JD, Wiseman MA (2008) The Eigenfactor metrics. J Neurosci 28(45):11433–11434.  https://doi.org/10.1523/JNEUROSCI.0003-08.2008 CrossRefPubMedGoogle Scholar
  24. 24.
    Garfield E (1955) Citation indexes for science; a new dimension in documentation through association of ideas. Science 122(3159):108–111CrossRefPubMedGoogle Scholar
  25. 25.
    Gisbert JP, Panes J (2009) The Hirsch's h-index: a new tool for measuring scientific production. Cirugia espanola 86(4):193–195.  https://doi.org/10.1016/j.ciresp.2009.05.007 CrossRefPubMedGoogle Scholar
  26. 26.
    Quindos G (2009) Confusing the confused: thoughts on impact factor, h(irsch) index, Q value, and other cofactors that influence the researcher's happiness. Revista iberoamericana de micologia : organo de la Asociacion Espanola de Especialistas en Micologia 26(2):97–102.  https://doi.org/10.1016/S1130-1406(09)70018-X CrossRefGoogle Scholar
  27. 27.
    Kianifar H, Sadeghi R, Zarifmahmoudi L (2014) Comparison between impact factor, eigenfactor metrics, and SCImago journal rank indicator of pediatric neurology journals. Acta informatica medica : AIM : journal of the Society for Medical Informatics of Bosnia & Herzegovina : casopis Drustva za medicinsku informatiku BiH 22(2):103–106.  https://doi.org/10.5455/aim.2014.22.103-106 CrossRefGoogle Scholar
  28. 28.
    Ascaso FJ (2011) Impact factor, eigenfactor and article influence. Archivos de la Sociedad Espanola de Oftalmologia 86(1):1–2.  https://doi.org/10.1016/j.oftal.2010.12.005 CrossRefPubMedGoogle Scholar
  29. 29.
    Seglen PO (1997) Why the impact factor of journals should not be used for evaluating research. BMJ 314(7079):498–502CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Marashi SA (2005) On the identity of "citers": are papers promptly recognized by other investigators? Med Hypotheses 65(4):822.  https://doi.org/10.1016/j.mehy.2005.05.003 CrossRefPubMedGoogle Scholar
  31. 31.
    Weale AR, Bailey M, Lear PA (2004) The level of non-citation of articles within a journal as a measure of quality: a comparison to the impact factor. BMC Med Res Methodol 4:14.  https://doi.org/10.1186/1471-2288-4-14 CrossRefPubMedPubMedCentralGoogle Scholar
  32. 32.
    Callaham M, Wears RL, Weber E (2002) Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA 287(21):2847–2850CrossRefPubMedGoogle Scholar
  33. 33.
    Garfield E (2000) Use of journal citation reports and journal performance indicators in measuring short and long term journal impact. Croatian medical journal 41(4):368–374PubMedGoogle Scholar
  34. 34.
    Van Noorden R (2016) Controversial impact factor gets a heavyweight rival. Nature 540(7633):325–326.  https://doi.org/10.1038/nature.2016.21131 CrossRefPubMedGoogle Scholar
  35. 35.
    Atayero AA, Popoola SI, Egeonu J, Oludayo O (2018) Citation analytics: data exploration and comparative analyses of CiteScores of open access and subscription-based publications indexed in Scopus (2014-2016). Data Brief 19:198–213.  https://doi.org/10.1016/j.dib.2018.05.005 CrossRefPubMedPubMedCentralGoogle Scholar
  36. 36.
    Hans Zijlstra RM (2016) CiteScore: a new metric to help you track journal performance and make decisions. https://www.elsevier.com/editors-update/story/journal-metrics/citescore-a-new-metric-to-help-you-choose-the-right-journal
  37. 37.
    Hale L (2017) Sleep Health receives its first CiteScore. Sleep Health 3(4):225.  https://doi.org/10.1016/j.sleh.2017.06.002 CrossRefPubMedGoogle Scholar
  38. 38.
    Sterbenc A, Ostrbenk A (2017) Elsevier's CiteScore index values for Acta Dermatovenerologica Alpina, Pannonica et Adriatica: a 2016 update. Acta Dermatovenerol Alp Pannonica Adriat 26(3):53PubMedGoogle Scholar
  39. 39.
    Vanden Eynde JJ (2017) Pharmaceuticals: impact factor or CiteScore. Pharmaceuticals (Basel) 10 (3). doi: https://doi.org/10.3390/ph10030061 CrossRefPubMedCentralGoogle Scholar
  40. 40.
    La Torre G, Sciarra I, Chiappetta M, Monteduro A (2017) New bibliometric indicators for the scientific literature: an evolving panorama. Clin Ter 168(2):e65–e71.  https://doi.org/10.7417/CT.2017.1985 CrossRefPubMedGoogle Scholar
  41. 41.
    Avena MJ, Barbosa DA (2017) Bibliometric indicators of the nursing journals according to the index databases. Rev Esc Enferm USP 51:e03262.  https://doi.org/10.1590/s1980-220x2017014603262 CrossRefPubMedGoogle Scholar
  42. 42.
    González-Pereira B, Guerrero-Bote VP, Moya-Anegón F (2010) A new approach to the metric of journals’ scientific prestige: the SJR indicator. Journal of Informetrics 4(3):379–391.  https://doi.org/10.1016/j.joi.2010.03.002 CrossRefGoogle Scholar
  43. 43.
    Newman MEJ (2010) Networks: an introduction, 1st. edn. Oxford University Press Inc., New YorkCrossRefGoogle Scholar
  44. 44.
    SCImago (2007) SJR—SCImago Journal & country rank. http://www.scimagojr.com. Accessed Retrieved 25 Nov 2013
  45. 45.
    Oosthuizen JC, Fenton JE (2014) Alternatives to the impact factor. Surgeon 12(5):239–243.  https://doi.org/10.1016/j.surge.2013.08.002 CrossRefPubMedGoogle Scholar
  46. 46.
    ELSEVIER (2018) Source-Normalized Impact per Paper (SNIP). https://journalinsights.elsevier.com/journals/0969-806X/snip
  47. 47.
    Kim K, Chung Y (2018) Overview of journal metrics. Sci Educ 5(1):16–20.  https://doi.org/10.6087/kcse.112 CrossRefGoogle Scholar
  48. 48.
    Moed HF (2010) Measuring contextual citation impact of scientific journals. J Inf Secur 4(3):265–277Google Scholar
  49. 49.
    Elsevier BV (2013) Source Normalized Impact per Paper. http://www.elsevier.com/editors/journal-metrics#metrics-in-scopus. Accessed 24 Nov 2103
  50. 50.
    Bergstrom CT, West JD (2008) Assessing citations with the eigenfactor metrics. Neurology 71(23):1850–1851.  https://doi.org/10.1212/01.wnl.0000338904.37585.66 CrossRefPubMedGoogle Scholar
  51. 51.
    von Bohlen Und Halbach O (2011) How to judge a book by its cover? How useful are bibliometric indices for the evaluation of "scientific quality" or "scientific productivity"? Ann Anat 193(3):191–196.  https://doi.org/10.1016/j.aanat.2011.03.011 CrossRefGoogle Scholar
  52. 52.
    Roldan-Valadez E, Rios C (2015) Alternative bibliometrics from impact factor improved the esteem of a journal in a 2-year-ahead annual-citation calculation: multivariate analysis of gastroenterology and hepatology journals. Eur J Gastroenterol Hepatol 27(2):115–122.  https://doi.org/10.1097/MEG.0000000000000253 CrossRefPubMedGoogle Scholar
  53. 53.
    Diaz-Ruiz A, Orbe-Arteaga U, Rios C, Roldan-Valadez E (2018) Alternative bibliometrics from the web of knowledge surpasses the impact factor in a 2-year ahead annual citation calculation: linear mixed-design models' analysis of neuroscience journals. Neurol India 66(1):96–104.  https://doi.org/10.4103/0028-3886.222880 CrossRefPubMedGoogle Scholar
  54. 54.
    Roldan-Valadez E, Orbe-Arteaga U, Rios C (2018) Eigenfactor score and alternative bibliometrics surpass the impact factor in a 2-years ahead annual-citation calculation: a linear mixed design model analysis of radiology, nuclear medicine and medical imaging journals. Radiol Med 123(7):524–534.  https://doi.org/10.1007/s11547-018-0870-y CrossRefPubMedGoogle Scholar
  55. 55.
    Bartneck C, Kokkelmans S (2011) Detecting h-index manipulation through self-citation analysis. Scientometrics 87(1):85–98.  https://doi.org/10.1007/s11192-010-0306-5 CrossRefPubMedGoogle Scholar
  56. 56.
    Ouimet M, Bedard PO, Gelineau F (2011) Are the H-index and some of its alternatives discriminatory of epistemological beliefs and methodological preferences of faculty members? The case of social scientists in Quebec. Scientometrics 88(1):91–106.  https://doi.org/10.1007/s11192-011-0364-3 CrossRefPubMedPubMedCentralGoogle Scholar
  57. 57.
    Purvis A (2006) The H index: playing the numbers game. Trends Ecol Evol 21(8):422.  https://doi.org/10.1016/j.tree.2006.05.014 CrossRefPubMedGoogle Scholar
  58. 58.
    Fernandez-Llimos F (2018) Differences and similarities between journal impact factor and CiteScore. Pharm Pract (Granada) 16(2):1282.  https://doi.org/10.18549/PharmPract.2018.02.1282 CrossRefGoogle Scholar
  59. 59.
    Agarwal A, Durairajanayagam D, Tatagari S, Esteves SC, Harlev A, Henkel R, Roychoudhury S, Homa S, Puchalt NG, Ramasamy R, Majzoub A, Ly KD, Tvrda E, Assidi M, Kesari K, Sharma R, Banihani S, Ko E, Abu-Elmagd M, Gosalvez J, Bashiri A (2016) Bibliometrics: tracking research impact by selecting the appropriate metrics. Asian journal of andrology 18(2):296–309.  https://doi.org/10.4103/1008-682x.171582 CrossRefPubMedPubMedCentralGoogle Scholar
  60. 60.
    Cooper ID (2015) Bibliometrics basics. Journal of the Medical Library Association : JMLA 103(4):217–218.  https://doi.org/10.3163/1536-5050.103.4.013 CrossRefPubMedGoogle Scholar
  61. 61.
    Garcia-Pachon E, Arencibia-Jorge R (2014) A comparison of the impact factor and the SCImago journal rank index in respiratory system journals. Arch bronconeumol 50(7):308–309.  https://doi.org/10.1016/j.arbres.2013.10.006 CrossRefPubMedGoogle Scholar
  62. 62.
    Hirsch JE (2005) An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A 102(46):16569–16572.  https://doi.org/10.1073/pnas.0507655102 CrossRefPubMedPubMedCentralGoogle Scholar
  63. 63.
    Salgado JF, Paez D (2007) Scientific productivity and Hirsch's h index of Spanish social psychology: convergence between productivity indexes and comparison with other areas. Psicothema 19(2):179–189PubMedGoogle Scholar
  64. 64.
    Wikipedia (2011) H-index. WikipediaGoogle Scholar
  65. 65.
    Hirsch JE (2007) Does the H index have predictive power? Proc Natl Acad Sci U S A 104(49):19193–19198.  https://doi.org/10.1073/pnas.0707962104 CrossRefPubMedPubMedCentralGoogle Scholar
  66. 66.
    Kulasegarah J, Fenton JE (2010) Comparison of the H index with standard bibliometric indicators to rank influential otolaryngologists in Europe and North America. Eur Arch Otorhinolaryngol 267(3):455–458.  https://doi.org/10.1007/s00405-009-1009-5 CrossRefPubMedGoogle Scholar
  67. 67.
    Iglesias JE, Pecharromán C (2007) Scaling the h-index for different scientific ISI field. Scientometrics 73(3):303–320CrossRefGoogle Scholar
  68. 68.
    Harzing AW (2007) Publish or perish, . Available from:http://www.harzing.com/pop.htm
  69. 69.
    Egghe L (2006) Theory and practise of the g-index. Scientometrics 69(1):131–152.  https://doi.org/10.1007/s11192-006-0144-7 CrossRefGoogle Scholar
  70. 70.
    Woeginger GJ (2008) An axiomatic analysis of Egghe’s g-index. Journal of Informetrics 2:364–368.  https://doi.org/10.1016/j.joi.2008.05.002 CrossRefGoogle Scholar
  71. 71.
    Serenko A (2010) The development of an AI journal ranking based on the revealed preference approach. Journal of Informetrics 4(4):447–459.  https://doi.org/10.1016/j.joi.2010.04.001 CrossRefGoogle Scholar
  72. 72.
    Sidiropoulos A, Katsaros D, Manolopoulos Y (2007) Generalized h-index for disclosing latent facts in citation networks. Scientometrics 72(2):253–280CrossRefGoogle Scholar
  73. 73.
    Zhang CT (2009) The e-index, complementing the h-index for excess citations. PLoS One 4(5):e5429.  https://doi.org/10.1371/journal.pone.0005429 CrossRefPubMedPubMedCentralGoogle Scholar
  74. 74.
    Dodson MV (2009) Citation analysis: maintenance of h-index and use of e-index. Biochem Biophys Res Commun 387(4):625–626.  https://doi.org/10.1016/j.bbrc.2009.07.091 CrossRefPubMedGoogle Scholar
  75. 75.
    Jin B (2007) The AR-index: complementing the H-index. ISSI Newsletter 3(1):6Google Scholar
  76. 76.
    Schreiber M (2008) To share the fame in a fair way, HM modifies H for multi-authored manuscripts. New J Phys 10(040201):9.  https://doi.org/10.1088/1367-2630/10/4/040201 CrossRefGoogle Scholar
  77. 77.
    Ruiz MA (2014) Goals, globalization and the impact factor of the journal. Rev Bras Hematol Hemoter 36(2):93–95.  https://doi.org/10.5581/1516-8484.20140019 CrossRefPubMedPubMedCentralGoogle Scholar
  78. 78.
    Olff M (2014) Are we happy with the impact factor? Eur J Psychotraumatol 5:26084.  https://doi.org/10.3402/ejpt.v5.26084 CrossRefGoogle Scholar
  79. 79.
    Altmetric (2013) What does altmetric do? http://support.altmetric.com/knowledgebase/articles/83246-altmetric-for-scopus. Accessed 24 Nov 2013
  80. 80.
    Elsevier BV (2013) Altmetrics. http://www.elsevier.com/editors/journal-metrics#metrics-in-scopus. Accessed 24 Nov 2013
  81. 81.
    Schmid SL (2017) Five years post-DORA: promoting best practices for research assessment. Mol Biol Cell 28(22):2941–2944.  https://doi.org/10.1091/mbc.E17-08-0534 CrossRefPubMedPubMedCentralGoogle Scholar
  82. 82.
    Science_Publishing_Group (2018) Declaration on research assessment. Copyright 2018 Science Publishing Group. https://spg.ltd/?s=DORA

Copyright information

© Royal Academy of Medicine in Ireland 2018

Authors and Affiliations

  1. 1.Directorate of ResearchHospital General de Mexico “Dr Eduardo Liceaga”Mexico CityMexico
  2. 2.Department of RadiologyI.M. Sechenov First Moscow State Medical University (Sechenov University)MoscowRussia
  3. 3.General Directorate of LibrariesNational Autonomous University of MexicoMexico CityMexico
  4. 4.Department of NeurochemistryNational Institute of Neurology and NeurosurgeryMexico CityMexico

Personalised recommendations