Abstract
Introduction
Because we believe the journal selection before a manuscript submission deserves further investigation in each medical specialty, we aimed to evaluate the predictive ability of seven bibliometrics in the Radiology, Nuclear Medicine and Medical Imaging category of the Web of Knowledge to calculate total citations over a 7-year period.
Methods
A linear mixed effects design using random slopes and intercepts were performed on bibliometrics corresponding to 124 journals from 2007 to 2011, with their corresponding citations from 2009 to 2013, which appeared in the Journal Citations Report Science Edition.
Results
The Eigenfactor Score, Article Influence Score, Cited Half-life, 5-years impact factor and Number of Articles are significant predictors of 2-year-ahead total citations (p ≤ 0.010 for all variables). The impact factor and Immediacy Index are not significant predictors. There was a significant global effect size (R2 = 0.934; p < 0.001), which yielded a total variance of 93.4%.
Conclusions
Our findings support researchers’ decision to stop the misuse of IF alone to evaluate journals. Radiologists and other researchers should review journal’s bibliometrics for their decision-making during the manuscript submission phase. A re-ranking of journals using Eigenfactor Score, Article Influence Score, and Cited Half-life provides a better assessment of their significance and importance in particular disciplines.
Similar content being viewed by others
References
Sharma M, Sarin A, Gupta P, Sachdeva S, Desai AV (2014) Journal impact factor: its use, significance and limitations. World J Nucl Med 13(2):146. https://doi.org/10.4103/1450-1147.139151
Kumar V, Upadhyay S, Medhi B (2009) Impact of the impact factor in biomedical research: its use and misuse. Singapore Med J 50(8):752–755
Baethge C (2012) Impact factor—a useful tool, but not for all purposes. Deutsches Arzteblatt Int 109(15):267–269. https://doi.org/10.3238/arztebl.2012.0267
Rawat S (2014) How is impact factor impacting our research? Biomed J. https://doi.org/10.4103/2319-4170.131388
Elliott DB (2014) The impact factor: a useful indicator of journal quality or fatally flawed? Ophthalmic Physiol Optics J Br Coll Ophthalmic Opt 34(1):4–7. https://doi.org/10.1111/opo.12107
Callaway E (2016) Beat it, impact factor! Publishing elite turns against controversial metric. Nature 535(7611):210–211. https://doi.org/10.1038/nature.2016.20224
Jackson A (2010) The impact factor game: the rising impact factor of the British Journal of Radiology—a success story? Br J Radiol 83(986):93–98. https://doi.org/10.1259/bjr/18689409
Ending the tyranny of the impact factor (2014) Nat Cell Biol 16(1):1. https://doi.org/10.1038/ncb2905
Diamandis EP (2017) The Journal Impact Factor is under attack—use the CAPCI factor instead. BMC Med 15(1):9. https://doi.org/10.1186/s12916-016-0773-5
Choudhri AF, Siddiqui A, Khan NR, Cohen HL (2015) Understanding bibliometric parameters and analysis. Radiographics 35(3):736–746. https://doi.org/10.1148/rg.2015140036
Sardanelli F, Sconfienza LM (2013) Declining impact factor of radiologic journals: a matter for debate. AJR Am J Roentgenol 201(3):W391–W393. https://doi.org/10.2214/AJR.12.10256
Shanta A, Pradhan AS, Sharma SD (2013) Impact factor of a scientific journal: is it a measure of quality of research? J Med Phys Assoc Med Phys India 38(4):155–157. https://doi.org/10.4103/0971-6203.121191
Thomson_Reuters (2014) Web of knowledge. http://wokinfo.com. Accessed 19 May 19 2014
Roldan-Valadez E, Rios C (2015) Alternative bibliometrics from impact factor improved the esteem of a journal in a 2-year-ahead annual-citation calculation: multivariate analysis of gastroenterology and hepatology journals. Eur J Gastroenterol Hepatol 27(2):115–122. https://doi.org/10.1097/MEG.0000000000000253
Diaz-Ruiz A, Orbe-Arteaga U, Rios C, Roldan-Valadez E (2018) Alternative bibliometrics from the web of knowledge surpasses the impact factor in a 2-year ahead annual citation calculation: linear mixed-design models’ analysis of neuroscience journals. Neurol India 66(1):96–104. https://doi.org/10.4103/0028-3886.222880
Garfield E (1996) How can impact factors be improved? BMJ 313(7054):411–413
Mathur VP, Sharma A (2009) Impact factor and other standardized measures of journal citation: a perspective. Indian J Dent Res Off Publ Indian Soc Dent Res 20(1):81–85
Bergstrom CT, West JD (2008) Assessing citations with the Eigenfactor metrics. Neurology 71(23):1850–1851. https://doi.org/10.1212/01.wnl.0000338904.37585.66
Bergstrom CT, West JD, Wiseman MA (2008) The Eigenfactor metrics. J Neurosci Off J Soc Neurosci 28(45):11433–11434. https://doi.org/10.1523/JNEUROSCI.0003-08.2008
Ascaso FJ (2011) Impact factor, eigenfactor and article influence. Archivos de la Sociedad Espanola de Oftalmologia 86(1):1–2. https://doi.org/10.1016/j.oftal.2010.12.005
Tabachnick BG, Fidell LS (2013) Multivariate normality. In: Tabachnick BG, Fidell LS (eds) Using multivariate statistics, 6th edn. p 253
Peugh JL (2010) A practical guide to multilevel modeling. J Sch Psychol 48(1):85–112. https://doi.org/10.1016/j.jsp.2009.09.002
Shek DT, Ma CM (2011) Longitudinal data analyses using linear mixed models in SPSS: concepts, procedures and illustrations. Sci World J 11:42–76. https://doi.org/10.1100/tsw.2011.2
Field A (2013) Multilevel Linear Models. In: Field A (ed) Discovering Statistics using SPSS, 4th edn. SAGE Publications Inc., London, pp 734–776
Pallant J (2011) Multiple regression. SPSS survival manual. Allen & Unwin, Crows Nest, pp 148–167
Cohen JW (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Lawrence Erlbaum Associates, Hillsdale
Singer JD (1998) Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. J Educ Behav Stat 23(4):323–355. https://doi.org/10.3102/10769986023004323
Butler JS, Sebastian AS, Kaye ID, Wagner SC, Morrissey PB, Schroeder GD, Kepler CK, Vaccaro AR (2017) Understanding traditional research impact metrics. Clin Spine Surg 30(4):164–166. https://doi.org/10.1097/BSD.0000000000000530
Reider B (2017) Brace for impact. Am J Sports Med 45(10):2213–2216. https://doi.org/10.1177/0363546517721707
Gutierrez FR, Beall J, Forero DA (2015) Spurious alternative impact factors: the scale of the problem from an academic perspective. BioEssays 37(5):474–476. https://doi.org/10.1002/bies.201500011
Citrome L (2013) How we rate: is impact factor the most important measure? Int J Clin Pract 67(9):819–820. https://doi.org/10.1111/ijcp.12266
Zupanc GK (2014) Impact beyond the impact factor. J Compar Physiol A Neuroethol Sens Neural Behav Physiol 200(2):113–116. https://doi.org/10.1007/s00359-013-0863-1
Kodumuri P, Ollivere B, Holley J, Moran CG (2014) The impact factor of a journal is a poor measure of the clinical relevance of its papers. Bone Joint J 96-B(3):414–419. https://doi.org/10.1302/0301-620x.96b3.32279
Sillet A, Katsahian S, Range H, Czernichow S, Bouchard P (2012) The Eigenfactor Score in highly specific medical fields: the dental model. J Dent Res 91(4):329–333. https://doi.org/10.1177/0022034512437374
Rizkallah J, Sin DD (2010) Integrative approach to quality assessment of medical journals using impact factor, eigenfactor, and article influence scores. PLoS One 5(4):e10204. https://doi.org/10.1371/journal.pone.0010204
Oosthuizen JC, Fenton JE (2014) Alternatives to the impact factor. Surgeon 12(5):239–243. https://doi.org/10.1016/j.surge.2013.08.002
Yao L, Wei T, Zeng A, Fan Y, Di Z (2014) Ranking scientific publications: the effect of nonlinearity. Sci Rep 4:6663. https://doi.org/10.1038/srep06663
Fersht A (2009) The most influential journals: impact Factor and Eigenfactor. Proc Natl Acad Sci USA 106(17):6883–6884. https://doi.org/10.1073/pnas.0903307106
Weale AR, Bailey M, Lear PA (2004) The level of non-citation of articles within a journal as a measure of quality: a comparison to the impact factor. BMC Med Res Methodol 4:14. https://doi.org/10.1186/1471-2288-4-14
Casadevall A, Fang FC (2014) Causes for the persistence of impact factor mania. mBio 5(2):e00064–e00014. https://doi.org/10.1128/mbio.00064-14
Chen C (2018) Opportunities and pitfalls in clinical proof-of-concept: principles and examples. Drug Discov Today. https://doi.org/10.1016/j.drudis.2018.01.045
Acknowledgements
Ulises Orbe-Arteaga, MSc, was a research fellow at the MRI unit of Medica Sur Clinic and Foundation during 2014–2015. No IRB approval was required for this manuscript.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have no conflict of interests to disclose.
Ethical standards
This article does not contain any studies with human participants or animals performed by any of the authors.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Roldan-Valadez, E., Orbe-Arteaga, U. & Rios, C. Eigenfactor score and alternative bibliometrics surpass the impact factor in a 2-years ahead annual-citation calculation: a linear mixed design model analysis of Radiology, Nuclear Medicine and Medical Imaging journals. Radiol med 123, 524–534 (2018). https://doi.org/10.1007/s11547-018-0870-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11547-018-0870-y