, Volume 114, Issue 3, pp 905–918 | Cite as

A study on the citation situation within the citing paper: citation distribution of references according to mention frequency

  • CholMyong Pak
  • Guang Yu
  • Weibin Wang


Citation impact indicators play a significant role in evaluating the scientific research activity. Most of citation impact indicators are based on the citation count that the publication is cited as a reference in the other publications, but the difference between each citation situation was not considered. Normally, the number of citations that a publication is cited in the other publications may represent the formal quality of the publication. Similarly, the number of times that a publication is really mentioned within the citing publication, it may also represent the formal quality of the citation. We have examined about how many times each reference was really mentioned within the citing publications and studied about the citation situation within the citing publications. We verified that the citation distribution of references according to the mention frequency follows the Generalized Pareto distribution. The results showed that about 20% of total references were mentioned three and more times, and the number of citation mentions for the about 50% of total references were from about 20% of the total references in the citing publications.


Citation analysis Citation impact indicator Generalized pareto distribution 80/20 rule 



This work was supported by the National Natural Science Foundation of China (Grant Nos. 71531013 and 71704035). Authors would like to express sincere thanks to the editors and reviewers of this paper.


  1. Aksnes, D. W. (2003). A macro study of self-citation. Scientometric, 56(2), 235–246.CrossRefGoogle Scholar
  2. Arnold, D. N. (2009). Integrity under attack: The state of scholarly publishing. SIAM News, 42(10), 1–3.Google Scholar
  3. Bertin, M., Atanassova, I., Gingras, Y., & Larivière, V. (2016). The invariant distribution of references in scientific articles. Journal of the Association forInformation Science and Technology, 67(1), 164–177.CrossRefGoogle Scholar
  4. Bollen, J., Rodriguez, M. A., & Van de Sompel, H. (2006). Journal status. Scientometrics, 69(3), 669–687.CrossRefGoogle Scholar
  5. Bornmann, L., & Daniel, H. D. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation, 64(1), 45–80.CrossRefGoogle Scholar
  6. Bornmann, L., & Haunschild, R. (2016). Citation score normalized by cited references (CSNCR): The introduction of a new citation impact indicator. Journal of Informetrics, 10, 875–887.CrossRefGoogle Scholar
  7. Buela-Casal, G. (2004). Assessing the quality of articles and scientific journals: Proposal for weighted impact factor. Psychology in Spain, 8(1), 60–76.Google Scholar
  8. Case, D. O., & Higgins, G. M. (2000). How can we investigate citation behavior? A study of reasons for citing literature in communication. Journal of the American Society for Information Science, 51(7), 635–645.CrossRefGoogle Scholar
  9. Ding, Y., Liu, X., Guo, C., & Cronin, B. (2013). The distribution of references across texts: Some implications for citation analysis. Journal of Informetrics, 7(3), 583–592.CrossRefGoogle Scholar
  10. Garfield, E. (1955). Citation indexes for science: A new dimension in documentation through association of ideas. Science, 122(3159), 108–111.CrossRefGoogle Scholar
  11. Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178(4060), 471–479.CrossRefGoogle Scholar
  12. Garfield, E. (1996). How can impact factors be improved? British Medical Journal, 313, 411–413.CrossRefGoogle Scholar
  13. Garfield, E. (2006). The history and meaning of the journal impact factor. Journal of the American Medical Association, 295(1), 90–93.CrossRefGoogle Scholar
  14. González-Pereira, B., Guerrero-Bote, V. P., & Moya-Anegón, F. (2010). A new approach to the metric of journals’ scientific prestige: The SJR indicator. Journalof Informetrics, 4(3), 379–391.CrossRefGoogle Scholar
  15. Guerrero-Bote, V. P., & Moya-Anegón, F. (2012). A further step forward in measuring journals’ scientific prestige: The SJR2 indicator. Journal of Informetrics, 6(4), 674–688.CrossRefGoogle Scholar
  16. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.CrossRefMATHGoogle Scholar
  17. Hirsch, J. E. (2010). An index to quantify an individual’s scientific research output that takes into account the effect of multiple co-authorship. Scientometrics, 85(3), 741–754.CrossRefGoogle Scholar
  18. Holsapple, C. W., Johnson, L. E., Manakyan, H., & Tanner, J. (1995). An empirical assessment and categorization of journals relevant to DSS research. Decision Support Systems, 14, 359–367.CrossRefGoogle Scholar
  19. Hu, Z., Chen, C., & Liu, Z. (2013). Where are citations located in the body of scientific articles? A study of the distributions of citation locations. Journal of Informetrics, 7(4), 887–896.CrossRefGoogle Scholar
  20. Hu, Z., Chen, C., Liu, Z. (2015). The Recurrence of Citations within a Scientific Article. In 15th international society of scientometrics and informetrics conference (pp. 221–229).Google Scholar
  21. Maslov, S., & Redner, S. (2008). Promise and pitfalls of extending Google’s PageRank algorithm to citation networks. Journal of Neuroscience, 28(44), 11103–11105.CrossRefGoogle Scholar
  22. Massey, F. J. (1951). The Kolmogorov-Smirnov test for goodness of fit. Journal of the American Statistical Association, 46(253), 68–78.CrossRefMATHGoogle Scholar
  23. Narin, F. (1976). Evaluative bibliometrics: The use of publication and citation analysis in the evaluation of scientific activity (pp. 184–198). Washington, DC: Computer Horizons.Google Scholar
  24. Nisonger, T. E. (2008). The “80/20 rule” and core journals. The Serials Librarian, 55, 62–84.CrossRefGoogle Scholar
  25. Opthof, T. (2013). Inflation of impact factors by journal self-citation in cardiovascular science. Netherlands Heart Journal, 21, 163–165.CrossRefGoogle Scholar
  26. Pickands, J. (1975). Statistical inference using extreme order statistics. The Annals of Statistics, 3(1), 119–131.MathSciNetCrossRefMATHGoogle Scholar
  27. Thelwall, M. (2016). Are the discretised lognormal and hooked power law distributions plausible for citation data. Journal of Informetrics, 10(2), 454–470.CrossRefGoogle Scholar
  28. Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365–391.MathSciNetCrossRefGoogle Scholar
  29. Waltman, L., & van Eck, N. J. (2013). A systematic empirical comparison of different approaches for normalizing citation impact indicators. Journal ofInformetrics, 7(4), 833–849.CrossRefGoogle Scholar
  30. Waltman, L., van Eck, N. J., van Leeuwen, T. N., Visser, M. S., & van Raan, A. F. J. (2011). Towards a new crown indicator: some theoretical considerations. Journal of Informetrics, 5(1), 37–47.CrossRefGoogle Scholar
  31. Wan, X., & Liu, F. (2014). WL-index: Leveraging citation mention number to quantify an individual’s scientific impact. Journal of the Association for Information Science and Technology, 65(12), 2509–2517.CrossRefGoogle Scholar
  32. Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S., et al. (2015). The metric tide: Report of the independent review of the role of metrics in research assessment and management bristol. Stoke Gifford: Higher Education Funding Council for England (HEFCE).CrossRefGoogle Scholar
  33. Yang, S., Lu, W., Zhang, Z., Wei, B., & An, W. (2016). Amplifying scientific paper’s abstract by leveraging data-weighted reconstruction. Information Processing and Management, 52, 698–719.CrossRefGoogle Scholar
  34. Zhao, D., Cappello, A., & Johnston, L. (2017). Functions of uni- and multi-citations: implications for weighted citation analysis. Journal of Data and Information Science, 2(1), 51–69.CrossRefGoogle Scholar
  35. Zhu, X., Turney, P., Lemire, D., & Vellino, A. (2015). Measuring academic influence: Not all citations are equal. Journal of the Association for Information Scienceand Technology, 66(2), 408–427.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2017

Authors and Affiliations

  1. 1.School of ManagementHarbin Institute of TechnologyHarbinChina
  2. 2.Kim Il Sung UniversityPyongyangDemocratic People’s Republic of Korea

Personalised recommendations