Skip to main content
SpringerLink
Log in

The influence of disruption on evaluating the scientific significance of papers

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

With the exponential growth of the volume of scientific literatures, academic evaluation is becoming one of the important problems of scientometrics. Considering the influence of disruption of papers, in this paper, we propose a disruption based PageRank (DPRank) model to rank the scientific significance of scientific papers. The disruption value of a paper is calculated by the local network structure of the paper in the citation network. DPRank model treats each citation differently based on paper’s temporal information and disruption value. In DPRank model, the scores delivered from a paper’s upstream nodes decay exponentially with the time gap between the publication time and reference time, and the decay rate is proportional to the paper’s disruption value. We use the APS data set and some highly valued papers as benchmarks to evaluate the performance of the proposed DPRank model. The experimental results show that the DPRank model has a relatively better performance for both old and newly published papers, and can outperform other methods in identifying the benchmark papers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. http://journals.aps.org/datasets.

  2. http://journals.aps.org/prl/50years/milestones.

References

  • Bahmani, B., Kumar, R., Mahdian, M., Upfal, E. (2012). Pagerank on an evolving graph. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 24–32).

  • Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems, 30, 107–117.

    Article  Google Scholar 

  • Chen, P., Xie, H., Maslov, S., & Redner, S. (2007). Finding scientific gems with Google’s PageRank algorithm. Journal of Informetrics, 1(1), 8–15.

    Article  Google Scholar 

  • Ding, Y. (2011). Applying weighted PageRank to author citation networks. Journal of the American Society for Information Science and Technology, 62(2), 236–245.

    Article  Google Scholar 

  • Fortunato, S., Bergstrom, C. T., Boerner, K., et al. (2018). Science of science. Science, 359, eaao0185.

    Article  Google Scholar 

  • Funk, R. J., & Owen-Smith, J. (2017). A dynamic network measure of technological change. Management Science, 63(3), 791–817.

    Article  Google Scholar 

  • Garfield, E. (2006). The history and meaning of the journal impact factor. JAMA, 295(1), 90–93.

    Article  Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, 102(46), 16569–16572.

    Article  Google Scholar 

  • Kanellos, I., Vergoulis, T., Sacharidis, D., Dalamagas, T., & Vassiliou, Y. (2019). Impact-based ranking of scientific publications: A survey and experimental evaluation. IEEE Transactions on Knowledge and Data Engineering, 2019, 1.

    Google Scholar 

  • Lü, L., Medo, M., Yeung, C. H., Zhang, Y. C., Zhang, Z. K., & Zhou, T. (2012). Recommender systems. Physics Reports, 519(1), 1–49.

    Article  Google Scholar 

  • Mariani, M. S., & Lü, L. (2020). Network-based ranking in social systems: Three challenges. Journal of Physics: Complexity, 1(1), 011001.

    Google Scholar 

  • Mariani, M. S., Medo, M., & Zhang, Y. C. (2015). Ranking nodes in growing networks: When PageRank fails. Scientific Reports, 5, 16181.

    Article  Google Scholar 

  • Mariani, M. S., Medo, M., & Zhang, Y. C. (2016). Identification of milestone papers through time-balanced network centrality. Journal of Informetrics, 10(4), 1207–1223.

    Article  Google Scholar 

  • Mingers, J., & Leydesdorff, L. (2015). A review of theory and practice in scientometrics. European Journal of Operational Research, 246(1), 1–19.

    Article  Google Scholar 

  • Newman, M. E. J. (2009). The first-mover advantage in scientific publication. Europhysics Letters, 86(6), 68001.

    Article  Google Scholar 

  • Page, L. (1999). The pagerank citation ranking: Bringing order to the web (pp. 1–14). Stanford: Stanford InfoLab.

    Google Scholar 

  • Parolo, P. D. B., Pan, R. K., Ghosh, R., Huberman, B. A., Kaski, K., & Fortunato, S. (2015). Attention decay in science. Journal of Informetrics, 9(4), 734–745.

    Article  Google Scholar 

  • Rozenshtein, P., Gionis, A. (2016). Temporal pagerank. In Joint European conference on machine learning and knowledge discovery in databases (Vol. 2016, pp. 674–689). Cham: Springer.

  • Sarewitz, D. (2016). The pressure to publish pushes down quality. Nature, 533(7602), 147.

    Article  Google Scholar 

  • Sinatra, R., Wang, D., Deville, P., Song, C., & Barabási, A. L. (2016). Quantifying the evolution of individual scientific impact. Science, 354(6312), aaf5239.

    Article  Google Scholar 

  • Walker, D., Xie, H., Yan, K. K., & Maslov, S. (2007). Ranking scientific publications using a model of network traffic. Journal of Statistical Mechanics: Theory and Experiment, 06, P06010.

    Google Scholar 

  • Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365–391.

    Article  Google Scholar 

  • Wang, Y., Zeng, A., Fan, Y., & Di, Z. (2019). Ranking scientific publications considering the aging characteristics of citations. Scientometrics, 120(1), 155–166.

    Article  Google Scholar 

  • Wu, L., Wang, D., & Evans, J. A. (2019). Large teams develop and small teams disrupt science and technology. Nature, 566(7744), 378–382.

    Article  Google Scholar 

  • Xu, S., Mariani, M. S., Lü, L., & Medo, M. (2020). Unbiased evaluation of ranking metrics reveals consistent performance in science and technology citation data. Journal of Informetrics, 14(1), 101005.

    Article  Google Scholar 

  • Yan, E., & Ding, Y. (2009). Applying centrality measures to impact analysis: A coauthorship network analysis. Journal of the American Society for Information Science and Technology, 60(10), 2107–2118.

    Article  Google Scholar 

  • Yao, L., Wei, T., Zeng, A., Fan, Y., & Di, Z. (2014). Ranking scientific publications: The effect of nonlinearity. Scientific Reports, 4, 6663.s.

    Article  Google Scholar 

  • Zamani, M., Tejedor, A., Vogl, M., Kräutli, F., Valleriani, M., & Kantz, H. (2020). Evolution and transformation of early modern cosmological knowledge: A network study. Scientific Reports, 10(1), 1–15.

    Article  Google Scholar 

  • Zeng, A., Shen, Z., Zhou, J., Wu, J., Fan, Y., Wang, Y., & Stanley, H. E. (2017). The science of science: From the perspective of complex systems. Physics Reports, 714, 1–73.

    Article  MathSciNet  Google Scholar 

  • Zhang, S., Medo, M., Lü, L., & Mariani, M. S. (2019). The long-term impact of ranking algorithms in growing networks. Information Sciences, 488, 257–271.

    Article  Google Scholar 

  • Zhou, Y., Cheng, H., Li, Q., & Wang, W. (2020). Diversity of temporal influence in popularity prediction of scientific publications. Scientometrics, 123(1), 383–392.

    Article  Google Scholar 

  • Zhou, Y., Li, Q., Yang, X., & Cheng, H. (2021). Predicting the popularity of scientific publications by an age-based diffusion model. Journal of Informetrics, 15(4), 101177.

    Article  Google Scholar 

  • Zhou, Y., Wang, R., Zeng, A., & Zhang, Y. C. (2020). Identifying prize-winning scientists by a competition-aware ranking. Journal of Informetrics, 14(3), 101038.

    Article  Google Scholar 

  • Zhou, J., Zeng, A., Fan, Y., & Di, Z. (2016). Ranking scientific publications with similarity-preferential mechanism. Scientometrics, 106, 805–816.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Nature Science Foundation of China under Grant 61603340 and 61773348.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, People’s Republic of China

    Yanbo Zhou, Xin-Li Xu, Xu-Hua Yang & Qu Li

Authors
  1. Yanbo Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xin-Li Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xu-Hua Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanbo Zhou.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Y., Xu, XL., Yang, XH. et al. The influence of disruption on evaluating the scientific significance of papers. Scientometrics 127, 5931–5945 (2022). https://doi.org/10.1007/s11192-022-04505-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-022-04505-4

Keywords

Search

Navigation