Skip to main content
SpringerLink
Log in

Evaluating journal impact based on weighted citations

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Publishing at high-rank journals is a common objective to most researchers, and there’s a crucial need for a journal ranking system with universal recognition. This paper presents a quantitative approach to rank scientific journals. The approach, HR-PageRank, combines weighted PageRank according to author’s H-index, and relevance between citing and cited papers. The output of the proposed approach is compared against journal impact factor, H5-index, PageRank algorithm and China Computer Federation ranking list. The experiments of quantifying scholarly impact objectively are conducted in two real scholarly data sets: (1) Microsoft Academic Graph and (2) Digital Bibliography and Library Project. Our experimental results indicate that HR-PageRank algorithm outperforms the well-known PageRank algorithm in finding the influential journals according to Spearman’s rank correlation coefficient, discounted cumulated gain and the correlation C.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. http://aka.ms/academicgraph.

  2. http://dblp.uni-trier.de.

  3. https://www.scopus.com.

References

  • Agarwal, A., Durairajanayagam, D., Tatagari, S., Esteves, S. C., Harlev, A., Henkel, R., et al. (2016). Bibliometrics: Tracking research impact by selecting the appropriate metrics. Asian Journal of Andrology, 18(2), 296–309.

    Article  Google Scholar 

  • Aragón, A. M. (2013). A measure for the impact of research. Scientific Reports, 3(3), 1649–1649.

    Article  Google Scholar 

  • Bai, X., Zhang, J., Cui, H., Ning, Z. & Xia, F. (2016). PNCOIRank: Evaluating the impact of scholarly articles with positive and negative citations. In Proceedings of the 25th international conference companion on World Wide Web, international world wide web conferences steering committee (pp. 9–10).

  • Ball, P. (2006). Prestige is factored into journal ratings. Nature, 439(7078), 770–771.

    Article  Google Scholar 

  • Boldi, P., Santini, M. & Vigna, S. (2005). Pagerank as a function of the damping factor. In International conference on World Wide Web (pp. 557–566).

  • Bornmann, L., & Daniel, H.-D. (2009). The state of h index research. EMBO Reports, 10(1), 2–6.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H.-D. (2011). A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. Journal of Informetrics, 5(3), 346–359.

    Article  Google Scholar 

  • Catalini, C., Lacetera, N., & Oettl, A. (2015). The incidence and role of negative citations in science. Proceedings of the National Academy of Sciences, 112(45), 13823–13826.

    Article  Google Scholar 

  • Chavda, J., & Patel, A. (2016). Measuring research impact: Bibliometrics, social media, altmetrics, and the BJGP. The British Journal of General Practice, 66(642), e59–e61.

    Article  Google Scholar 

  • Chen, P., Xie, H., Maslov, S., & Redner, S. (2007). Finding scientific gems with Google’s PageRank algorithm. Journal of Informetrics, 1(1), 8–15.

    Article  Google Scholar 

  • Chen, Y.-L., & Chen, X.-H. (2011). An evolutionary pagerank approach for journal ranking with expert judgements. Journal of Information Science, 37(3), 254–272.

    Article  MathSciNet  Google Scholar 

  • Cohen, J. (1977). Statistical power analysis for the behavioral sciences (pp. 19–74). New York: Academy Press.

    Book  Google Scholar 

  • Egghe, L. (2006). Theory and practise of the g-index. Scientometrics, 69(1), 131–152.

    Article  Google Scholar 

  • Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178(4060), 471–479.

    Article  Google Scholar 

  • Garfield, E., & Sher, I. H. (1963). New factors in the evaluation of scientific literature through citation indexing. American Documentation, 14(3), 195–201.

    Article  Google Scholar 

  • Guimera, R., Uzzi, B., Spiro, J., & Amaral, L. A. N. (2005). Team assembly mechanisms determine collaboration network structure and team performance. Science, 308(5722), 697–702.

    Article  Google Scholar 

  • Harzing, A.-W., & Van Der Wal, R. (2009). A Google Scholar h-index for journals: An alternative metric to measure journal impact in economics and business. Journal of the American Society for Information Science and Technology, 60(1), 41–46.

    Article  Google Scholar 

  • Hirsch, J. (2010). An index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship. Scientometrics, 85(3), 741–754.

    Article  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National academy of Sciences of the United States of America, 102(46), 16569–16572.

    Article  MATH  Google Scholar 

  • Järvelin, K., & Kekäläinen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS), 20(4), 422–446.

    Article  Google Scholar 

  • Liu, Z., Huang, H., Wei, X. & Mao, X. (2014). Tri-rank: An authority ranking framework in heterogeneous academic networks by mutual reinforce. In 26th IEEE international conference on tools with artificial intelligence (ICTAI), IEEE (pp. 493–500).

  • Myers, J. L., Well, A., & Lorch, R. F. (2010). Research design and statistical analysis. London: Routledge.

    Google Scholar 

  • Pan, R. K., & Fortunato, S. (2014). Author impact factor: Tracking the dynamics of individual scientific impact. Scientific Reports, 4, 4880–4880.

    Article  Google Scholar 

  • Schreiber, M. (2015). A variant of the h-index to measure recent performance. Journal of the Association for Information Science and Technology, 66(11), 2373–2380.

    Article  Google Scholar 

  • Shah, N. & Song, Y. (2015). S-index: Towards better metrics for quantifying research impact. arXiv preprint arXiv:1507.03650.

  • Simkin, M., & Roychowdhury, V. (2003). Read before you cite!. Complex Systems, 14, 269–274.

    Google Scholar 

  • Stallings, J., Vance, E., Yang, J., Vannier, M. W., Liang, J., Pang, L., et al. (2013). Determining scientific impact using a collaboration index. Proceedings of the National Academy of Sciences, 110(24), 9680–9685.

    Article  MathSciNet  MATH  Google Scholar 

  • Su, C., Pan, Y., Zhen, Y., Ma, Z., Yuan, J., Guo, H., et al. (2011). PrestigeRank: A new evaluation method for papers and journals. Journal of Informetrics, 5(1), 1–13.

    Article  Google Scholar 

  • Valderas, J. M. (2007). Why do team-authored papers get cited more? Science, 317(5844), 1496–1498.

    Article  Google Scholar 

  • van Ypersele, J.-P., et al. (2013). The maze of impact metrics. Nature, 502(7472), 423–423.

    Article  Google Scholar 

  • Wang, D., Song, C., & Barabási, A.-L. (2013). Quantifying long-term scientific impact. Science, 342(6154), 127–132.

    Article  Google Scholar 

  • Wang, Y., Tong, Y. & Zeng, M. (2013). Ranking scientific articles by exploiting citations, authors, journals, and time information. In Proceedings of the twenty-seventh AAAI conference on artificial intelligence (pp. 933–939). AAAI Press.

  • Wasserman, M., Zeng, X. H. T., & Amaral, L. A. N. (2015). Cross-evaluation of metrics to estimate the significance of creative works. Proceedings of the National Academy of Sciences, 112(5), 1281–1286.

    Article  MathSciNet  MATH  Google Scholar 

  • Yan, E., & Ding, Y. (2011). Discovering author impact: A pagerank perspective. Information Processing & Management, 47(1), 125–134.

    Article  Google Scholar 

  • Yan, E., Ding, Y., & Sugimoto, C. R. (2011). P-Rank: An indicator measuring prestige in heterogeneous scholarly networks. Journal of the American Society for Information Science and Technology, 62(3), 467–477.

    Google Scholar 

  • Zhang, C.-T. (2013). A novel triangle mapping technique to study the h-index based citation distribution. Scientific Reports, 3, 1023–1023.

    Article  Google Scholar 

Download references

Acknowledgements

The author thanks Xiaomei Bai and Haozhen Liu (School of Software, Dalian University of Technology) for support in data processing and algorithm implementations.

Author information

Authors and Affiliations

  1. Library, Anshan Normal University, Anshan, 114007, China

    Fuli Zhang

Authors
  1. Fuli Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fuli Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, F. Evaluating journal impact based on weighted citations. Scientometrics 113, 1155–1169 (2017). https://doi.org/10.1007/s11192-017-2510-z

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-017-2510-z

Keywords

Search

Navigation