Citation regression analysis of computer science publications in different ranking categories and subfields

Abstract

A number of bibliometric studies point out that the role of conference publications in computer science differs from that in other traditional fields. Thus, it is interesting to identify the relative status of journal and conference publications in different subfields of computer science based on the citation rates categorised by the China Computer Federation (CCF) classifications and venue types. In this research, we construct a dataset containing over 100,000 papers recommended by the CCF catalogue and their citation information. We also investigate some other factors that often influence a paper’s citation rate. An experimental study shows that the relative status of journals and conferences varies greatly in different subfields of computer science, and the impact of different publication levels varies according to the citation rate. We also verify that the classification of a publication, number of authors, maximum h-index of all authors of a paper, and average number of papers published by a publication have different effects on the citation rate, although the citation rate may have a different degree of correlation with these factors.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. 1.

    http://www3.ntu.edu.sg/home/assourav/crank.html.

  2. 2.

    http://webdocs.cs.ualberta.ca/~zaiane/htmldocs/ConfRanking.html.

  3. 3.

    http://www.core.edu.au/.

  4. 4.

    http://www.ccf.org.cn/sites/ccf/paiming.jsp.

  5. 5.

    http://www.scimagojr.com/.

  6. 6.

    http://www.ccf.org.cn/sites/ccf/paiming.jsp.

  7. 7.

    https://cn.aminer.org/aminernetwork.

  8. 8.

    http://dblp.uni-trier.de/db/.

References

  1. Aksnes, D. W. (2003). Characteristics of highly cited papers. Research Evaluation, 12(3), 159–170.

    Article  Google Scholar 

  2. Beel, J., & Gipp, B. (2010). Academic search engine spam and google scholar’s resilience against it. Journal of Electronic Publishing, 13(3). http://quod.lib.umich.edu/cgi/t/text/text-idx?c=jep;view=text;rgn=main;idno=3336451.0013.305.

  3. Bensman, S. J., Smolinsky, L. J., & Pudovkin, A. I. (2010). Mean citation rate per article in mathematics journals: Differences from the scientific model. Journal of the American Society for Information Science and Technology, 61(7), 1440–1463.

    Article  Google Scholar 

  4. Berk, R., & MacDonald, J. M. (2008). Overdispersion and poisson regression. Journal of Quantitative Criminology, 24(3), 269–284.

    Article  Google Scholar 

  5. Bornmann, L., & Daniel, H. D. (2008). Selecting manuscripts for a high-impact journal through peer review: A citation analysis of communications that were accepted by angewandte chemie international edition, or rejected but published elsewhere. Journal of the American Society for Information Science and Technology, 59(11), 1841–1852.

    Article  Google Scholar 

  6. Bornmann, L., Schier, H., Marx, W., & Daniel, H. D. (2012). What factors determine citation counts of publications in chemistry besides their quality? Journal of Informetrics, 6(1), 11–18.

    Article  Google Scholar 

  7. Bruin, J. (2006). Newtest: Command to compute new test. Los Angeles: UCLA: Academic Technology Services, Statistical Consulting Group.

    Google Scholar 

  8. Butler, D. (2008). Free journal-ranking tool enters citation market. Nature News, 451(7174), 6–6.

    Article  Google Scholar 

  9. Cameron, A. C., & Trivedi, P. K. (2013). Regression analysis of count data (Vol. 53). Cambridge: Cambridge University Press.

    Book  MATH  Google Scholar 

  10. Chen, J., & Konstan, J. A. (2010). Conference paper selectivity and impact. Communications of the ACM, 53(6), 79–83.

    Article  Google Scholar 

  11. Crespo, J. A., Herranz, N., Li, Y., & Ruiz-Castillo, J. (2014). The effect on citation inequality of differences in citation practices at the web of science subject category level. Journal of the Association for Information Science and Technology, 65(6), 1244–1256.

    Article  Google Scholar 

  12. Dalgaard, P. (2008). Rates and poisson regression. In Introductory statistics with R (pp. 259–274). Berlin: Springer.

  13. Didegah, F., & Thelwall, M. (2013). Determinants of research citation impact in nanoscience and nanotechnology. Journal of the American Society for Information Science and Technology, 64(5), 1055–1064.

    Article  Google Scholar 

  14. Eckmann, M., Rocha, A., & Wainer, J. (2011). Relationship between high-quality journals and conferences in computer vision. Scientometrics, 90(2), 617–630.

    Article  Google Scholar 

  15. Fairclough, R., & Thelwall, M. (2015). More precise methods for national research citation impact comparisons. Journal of Informetrics, 9(4), 895–906.

    Article  Google Scholar 

  16. Falagas, M. E., Kouranos, V. D., Arencibia-Jorge, R., & Karageorgopoulos, D. E. (2008). Comparison of scimago journal rank indicator with journal impact factor. The FASEB Journal, 22(8), 2623–2628.

    Article  Google Scholar 

  17. Fanelli, D. (2013). Positive results receive more citations, but only in some disciplines. Scientometrics, 94(2), 701–709.

    Article  Google Scholar 

  18. Fortnow, L. (2009). Viewpoint time for computer science to grow up. Communications of the ACM, 52(8), 33–35.

    Article  Google Scholar 

  19. Franceschet, M. (2010). The role of conference publications in CS. Communications of the ACM, 53(12), 129–132.

    Article  Google Scholar 

  20. Freyne, J., Coyle, L., Smyth, B., & Cunningham, P. (2010). Relative status of journal and conference publications in computer science. Communications of the ACM, 53(11), 124–132.

    Article  Google Scholar 

  21. Garfield, E. (2006). The history and meaning of the journal impact factor. Jama, 295(1), 90–93.

    Article  Google Scholar 

  22. He, Z. L. (2009). International collaboration does not have greater epistemic authority. Journal of the American Society for Information Science and Technology, 60(10), 2151–2164.

    Article  Google Scholar 

  23. Leimu, R., & Koricheva, J. (2005). What determines the citation frequency of ecological papers? Trends in Ecology & Evolution, 20(1), 28–32.

    Article  Google Scholar 

  24. Long, J. S., & Freese, J. (2006). Regression models for categorical dependent variables using Stata. College Station: Stata Press.

    MATH  Google Scholar 

  25. Lozano, G. A., Larivière, V., & Gingras, Y. (2012). The weakening relationship between the impact factor and papers’ citations in the digital age. Journal of the American Society for Information Science and Technology, 63(11), 2140–2145.

    Article  Google Scholar 

  26. Marx, W., & Bornmann, L. (2014). On the causes of subject-specific citation rates in web of science. Scientometrics, 102(2), 1823–1827.

    Article  Google Scholar 

  27. Onodera, N., & Yoshikane, F. (2015). Factors affecting citation rates of research articles. Journal of the Association for Information Science and Technology, 66(4), 739–764.

    Article  Google Scholar 

  28. Peng, T. Q., & Zhu, J. J. (2012). Where you publish matters most: A multilevel analysis of factors affecting citations of internet studies. Journal of the American Society for Information Science and Technology, 63(9), 1789–1803.

    Article  Google Scholar 

  29. Rahm, E., & Thor, A. (2005). Citation analysis of database publications. ACM Sigmod Record, 34(4), 48–53.

    Article  Google Scholar 

  30. Rigby, J. (2013). Looking for the impact of peer review: Does count of funding acknowledgements really predict research impact? Scientometrics, 94(1), 57–73.

    Article  Google Scholar 

  31. Seglen, P. O. (1994). Causal relationship between article citedness and journal impact. Journal of the American Society for Information Science, 45(1), 1–11.

    Article  Google Scholar 

  32. Shamir, L. (2010). The effect of conference proceedings on the scholarly communication in computer science and engineering. Scholarly and Research Communication, 1(2). http://www.srconline.ca/index.php/src/article/viewFile/25/43.

  33. Smolinsky, L. (2016). Expected number of citations and the crown indicator. Journal of Informetrics, 10(1), 43–47.

    Article  Google Scholar 

  34. StataCorp (2005). Stata Statistical Software Release 9. College Station: StataCorp LP.

  35. Tahamtan, I., Afshar, A. S., & Ahamdzadeh, K. (2016). Factors affecting number of citations: A comprehensive review of the literature. Scientometrics, 107(3), 1195–1225.

    Article  Google Scholar 

  36. Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., & Su, Z. (2008). Arnetminer: Extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining, pp 990–998.

  37. Thelwall, M., & Fairclough, R. (2015). The influence of time and discipline on the magnitude of correlations between citation counts and quality scores. Journal of Informetrics, 9(3), 529–541.

    Article  Google Scholar 

  38. Thelwall, M., & Wilson, P. (2014). Regression for citation data: An evaluation of different methods. Journal of Informetrics, 8(4), 963–971.

    Article  Google Scholar 

  39. Van Dalen, H., & Henkens, K. (2001). What makes a scientific article influential? The case of demographers. Scientometrics, 50(3), 455–482.

    Article  Google Scholar 

  40. Van Dalen, H. P., & Kn, H. (2005). Signals in science-on the importance of signaling in gaining attention in science. Scientometrics, 64(2), 209–233.

    Article  Google Scholar 

  41. Vardi, M. Y. (2009). Conferences vs. journals in computing research. Communications of the ACM, 52(5), 5.

    Article  Google Scholar 

  42. Vrettas, G., & Sanderson, M. (2015). Conferences versus journals in computer science. Journal of the Association for Information Science and Technology, 66(12), 2674–2684.

    Article  Google Scholar 

  43. Walters, G. D. (2006). Predicting subsequent citations to articles published in twelve crime-psychology journals: Author impact versus journal impact. Scientometrics, 69(3), 499–510.

    Article  Google Scholar 

  44. Wang, D., Song, C., & Barabási, A. L. (2013). Quantifying long-term scientific impact. Science, 342(6154), 127–132.

    Article  Google Scholar 

  45. Wang, M., Yu, G., & Yu, D. (2011). Mining typical features for highly cited papers. Scientometrics, 87(3), 695–706.

    Article  Google Scholar 

  46. Wang, M., Yu, G., An, S., & Yu, D. (2012). Discovery of factors influencing citation impact based on a soft fuzzy rough set model. Scientometrics, 93(3), 635–644.

    Article  Google Scholar 

  47. Zhu, X., Turney, P., Lemire, D., & Vellino, A. (2015). Measuring academic influence: Not all citations are equal. Journal of the Association for Information Science and Technology, 66(2), 408–427.

    Article  Google Scholar 

  48. Zitt, M. (2012). The journal impact factor: Angel, devil, or scapegoat? A comment on JK Vanclay’s article 2011. Scientometrics, 92(2), 485–503.

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially supported by the State Key Laboratory of Software Development Environment of China (No. SKLSDE-2017ZX-15), the National Social Science Foundation of China (No. 13&ZD190), an Royal Society-Newton Advanced Fellowship Award, and the Fundamental Research Funds for the Central Universities.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Wenge Rong.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Qian, Y., Rong, W., Jiang, N. et al. Citation regression analysis of computer science publications in different ranking categories and subfields. Scientometrics 110, 1351–1374 (2017). https://doi.org/10.1007/s11192-016-2235-4

Download citation

Keywords

  • Citation rate
  • Computer science
  • Influence factor
  • Multiple regression