, Volume 110, Issue 3, pp 1351–1374 | Cite as

Citation regression analysis of computer science publications in different ranking categories and subfields

  • Yifan Qian
  • Wenge RongEmail author
  • Nan Jiang
  • Jie Tang
  • Zhang Xiong


A number of bibliometric studies point out that the role of conference publications in computer science differs from that in other traditional fields. Thus, it is interesting to identify the relative status of journal and conference publications in different subfields of computer science based on the citation rates categorised by the China Computer Federation (CCF) classifications and venue types. In this research, we construct a dataset containing over 100,000 papers recommended by the CCF catalogue and their citation information. We also investigate some other factors that often influence a paper’s citation rate. An experimental study shows that the relative status of journals and conferences varies greatly in different subfields of computer science, and the impact of different publication levels varies according to the citation rate. We also verify that the classification of a publication, number of authors, maximum h-index of all authors of a paper, and average number of papers published by a publication have different effects on the citation rate, although the citation rate may have a different degree of correlation with these factors.


Citation rate Computer science Influence factor Multiple regression 



This work was partially supported by the State Key Laboratory of Software Development Environment of China (No. SKLSDE-2017ZX-15), the National Social Science Foundation of China (No. 13&ZD190), an Royal Society-Newton Advanced Fellowship Award, and the Fundamental Research Funds for the Central Universities.


  1. Aksnes, D. W. (2003). Characteristics of highly cited papers. Research Evaluation, 12(3), 159–170.CrossRefGoogle Scholar
  2. Beel, J., & Gipp, B. (2010). Academic search engine spam and google scholar’s resilience against it. Journal of Electronic Publishing, 13(3).;view=text;rgn=main;idno=3336451.0013.305.
  3. Bensman, S. J., Smolinsky, L. J., & Pudovkin, A. I. (2010). Mean citation rate per article in mathematics journals: Differences from the scientific model. Journal of the American Society for Information Science and Technology, 61(7), 1440–1463.CrossRefGoogle Scholar
  4. Berk, R., & MacDonald, J. M. (2008). Overdispersion and poisson regression. Journal of Quantitative Criminology, 24(3), 269–284.CrossRefGoogle Scholar
  5. Bornmann, L., & Daniel, H. D. (2008). Selecting manuscripts for a high-impact journal through peer review: A citation analysis of communications that were accepted by angewandte chemie international edition, or rejected but published elsewhere. Journal of the American Society for Information Science and Technology, 59(11), 1841–1852.CrossRefGoogle Scholar
  6. Bornmann, L., Schier, H., Marx, W., & Daniel, H. D. (2012). What factors determine citation counts of publications in chemistry besides their quality? Journal of Informetrics, 6(1), 11–18.CrossRefGoogle Scholar
  7. Bruin, J. (2006). Newtest: Command to compute new test. Los Angeles: UCLA: Academic Technology Services, Statistical Consulting Group.Google Scholar
  8. Butler, D. (2008). Free journal-ranking tool enters citation market. Nature News, 451(7174), 6–6.CrossRefGoogle Scholar
  9. Cameron, A. C., & Trivedi, P. K. (2013). Regression analysis of count data (Vol. 53). Cambridge: Cambridge University Press.CrossRefzbMATHGoogle Scholar
  10. Chen, J., & Konstan, J. A. (2010). Conference paper selectivity and impact. Communications of the ACM, 53(6), 79–83.CrossRefGoogle Scholar
  11. Crespo, J. A., Herranz, N., Li, Y., & Ruiz-Castillo, J. (2014). The effect on citation inequality of differences in citation practices at the web of science subject category level. Journal of the Association for Information Science and Technology, 65(6), 1244–1256.CrossRefGoogle Scholar
  12. Dalgaard, P. (2008). Rates and poisson regression. In Introductory statistics with R (pp. 259–274). Berlin: Springer.Google Scholar
  13. Didegah, F., & Thelwall, M. (2013). Determinants of research citation impact in nanoscience and nanotechnology. Journal of the American Society for Information Science and Technology, 64(5), 1055–1064.CrossRefGoogle Scholar
  14. Eckmann, M., Rocha, A., & Wainer, J. (2011). Relationship between high-quality journals and conferences in computer vision. Scientometrics, 90(2), 617–630.CrossRefGoogle Scholar
  15. Fairclough, R., & Thelwall, M. (2015). More precise methods for national research citation impact comparisons. Journal of Informetrics, 9(4), 895–906.CrossRefGoogle Scholar
  16. Falagas, M. E., Kouranos, V. D., Arencibia-Jorge, R., & Karageorgopoulos, D. E. (2008). Comparison of scimago journal rank indicator with journal impact factor. The FASEB Journal, 22(8), 2623–2628.CrossRefGoogle Scholar
  17. Fanelli, D. (2013). Positive results receive more citations, but only in some disciplines. Scientometrics, 94(2), 701–709.CrossRefGoogle Scholar
  18. Fortnow, L. (2009). Viewpoint time for computer science to grow up. Communications of the ACM, 52(8), 33–35.CrossRefGoogle Scholar
  19. Franceschet, M. (2010). The role of conference publications in CS. Communications of the ACM, 53(12), 129–132.CrossRefGoogle Scholar
  20. Freyne, J., Coyle, L., Smyth, B., & Cunningham, P. (2010). Relative status of journal and conference publications in computer science. Communications of the ACM, 53(11), 124–132.CrossRefGoogle Scholar
  21. Garfield, E. (2006). The history and meaning of the journal impact factor. Jama, 295(1), 90–93.CrossRefGoogle Scholar
  22. He, Z. L. (2009). International collaboration does not have greater epistemic authority. Journal of the American Society for Information Science and Technology, 60(10), 2151–2164.CrossRefGoogle Scholar
  23. Leimu, R., & Koricheva, J. (2005). What determines the citation frequency of ecological papers? Trends in Ecology & Evolution, 20(1), 28–32.CrossRefGoogle Scholar
  24. Long, J. S., & Freese, J. (2006). Regression models for categorical dependent variables using Stata. College Station: Stata Press.zbMATHGoogle Scholar
  25. Lozano, G. A., Larivière, V., & Gingras, Y. (2012). The weakening relationship between the impact factor and papers’ citations in the digital age. Journal of the American Society for Information Science and Technology, 63(11), 2140–2145.CrossRefGoogle Scholar
  26. Marx, W., & Bornmann, L. (2014). On the causes of subject-specific citation rates in web of science. Scientometrics, 102(2), 1823–1827.CrossRefGoogle Scholar
  27. Onodera, N., & Yoshikane, F. (2015). Factors affecting citation rates of research articles. Journal of the Association for Information Science and Technology, 66(4), 739–764.CrossRefGoogle Scholar
  28. Peng, T. Q., & Zhu, J. J. (2012). Where you publish matters most: A multilevel analysis of factors affecting citations of internet studies. Journal of the American Society for Information Science and Technology, 63(9), 1789–1803.CrossRefGoogle Scholar
  29. Rahm, E., & Thor, A. (2005). Citation analysis of database publications. ACM Sigmod Record, 34(4), 48–53.CrossRefGoogle Scholar
  30. Rigby, J. (2013). Looking for the impact of peer review: Does count of funding acknowledgements really predict research impact? Scientometrics, 94(1), 57–73.CrossRefGoogle Scholar
  31. Seglen, P. O. (1994). Causal relationship between article citedness and journal impact. Journal of the American Society for Information Science, 45(1), 1–11.CrossRefGoogle Scholar
  32. Shamir, L. (2010). The effect of conference proceedings on the scholarly communication in computer science and engineering. Scholarly and Research Communication, 1(2).
  33. Smolinsky, L. (2016). Expected number of citations and the crown indicator. Journal of Informetrics, 10(1), 43–47.CrossRefGoogle Scholar
  34. StataCorp (2005). Stata Statistical Software Release 9. College Station: StataCorp LP.Google Scholar
  35. Tahamtan, I., Afshar, A. S., & Ahamdzadeh, K. (2016). Factors affecting number of citations: A comprehensive review of the literature. Scientometrics, 107(3), 1195–1225.CrossRefGoogle Scholar
  36. Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., & Su, Z. (2008). Arnetminer: Extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining, pp 990–998.Google Scholar
  37. Thelwall, M., & Fairclough, R. (2015). The influence of time and discipline on the magnitude of correlations between citation counts and quality scores. Journal of Informetrics, 9(3), 529–541.CrossRefGoogle Scholar
  38. Thelwall, M., & Wilson, P. (2014). Regression for citation data: An evaluation of different methods. Journal of Informetrics, 8(4), 963–971.CrossRefGoogle Scholar
  39. Van Dalen, H., & Henkens, K. (2001). What makes a scientific article influential? The case of demographers. Scientometrics, 50(3), 455–482.CrossRefGoogle Scholar
  40. Van Dalen, H. P., & Kn, H. (2005). Signals in science-on the importance of signaling in gaining attention in science. Scientometrics, 64(2), 209–233.CrossRefGoogle Scholar
  41. Vardi, M. Y. (2009). Conferences vs. journals in computing research. Communications of the ACM, 52(5), 5.CrossRefGoogle Scholar
  42. Vrettas, G., & Sanderson, M. (2015). Conferences versus journals in computer science. Journal of the Association for Information Science and Technology, 66(12), 2674–2684.CrossRefGoogle Scholar
  43. Walters, G. D. (2006). Predicting subsequent citations to articles published in twelve crime-psychology journals: Author impact versus journal impact. Scientometrics, 69(3), 499–510.CrossRefGoogle Scholar
  44. Wang, D., Song, C., & Barabási, A. L. (2013). Quantifying long-term scientific impact. Science, 342(6154), 127–132.CrossRefGoogle Scholar
  45. Wang, M., Yu, G., & Yu, D. (2011). Mining typical features for highly cited papers. Scientometrics, 87(3), 695–706.CrossRefGoogle Scholar
  46. Wang, M., Yu, G., An, S., & Yu, D. (2012). Discovery of factors influencing citation impact based on a soft fuzzy rough set model. Scientometrics, 93(3), 635–644.CrossRefGoogle Scholar
  47. Zhu, X., Turney, P., Lemire, D., & Vellino, A. (2015). Measuring academic influence: Not all citations are equal. Journal of the Association for Information Science and Technology, 66(2), 408–427.CrossRefGoogle Scholar
  48. Zitt, M. (2012). The journal impact factor: Angel, devil, or scapegoat? A comment on JK Vanclay’s article 2011. Scientometrics, 92(2), 485–503.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2017

Authors and Affiliations

  1. 1.State Key Laboratory of Software Development EnvironmentBeihang UniversityBeijingChina
  2. 2.Sino-French Engineer SchoolBeihang UniversityBeijingChina
  3. 3.School of Computer Science and EngineeringBeihang UniversityBeijingChina
  4. 4.Department of Computer Science and TechnologyTsinghua UniversityBeijingChina

Personalised recommendations