Analysis of co-authorship graphs of CORE-ranked software conferences

Abstract

In most areas of computer science (CS), and in the software domain in particular, international conferences are as important as journals as a venue to disseminate research results. This has resulted in the creation of rankings to provide quality assessment of conferences (specially used for academic promotion purposes) like the well-known CORE ranking created by the Computing Research and Education Association of Australasia. In this paper we analyze 102 CORE-ranked conferences in the software area (covering all aspects of software engineering, programming languages, software architectures and the like) included in the DBLP dataset, an online reference for computers science bibliographic information. We define a suite of metrics focusing on the analysis of the co-authorship graph of the conferences, where authors are represented as nodes and co-authorship relationships as edges. Our aim is to first characterize the patterns and structure of the community of researchers in software conferences. We then try to see if these values depend on the quality rank of the conference justifying this way the existence of the different classifications in the CORE-ranking system.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Notes

  1. 1.

    Microsoft Academic Search Conference Ranking. http://bit.ly/1WMsg4y.

  2. 2.

    Simple H-Index Estimator. http://shine.icomp.ufam.edu.br/about.php.

  3. 3.

    Initiative sponsored by GII (Group of Italian Professors of Computer Engineering) and GRIN (Group of Italian Professors of Computer Science). http://valutazione.unibas.it/cs-conference-rating/conferenceRating.jsf.

  4. 4.

    http://www.core.edu.au.

  5. 5.

    A more detailed description about how the classification is defined can be found at http://www.core.edu.au/documents/RankingDescriptions2014.pdf.

  6. 6.

    Available at http://portal.core.edu.au/conf-ranks as CSV file.

  7. 7.

    http://dblp.uni-trier.de.

  8. 8.

    Available at http://dblp.uni-trier.de/xml.

  9. 9.

    Provided by http://dblp.l3s.de.

  10. 10.

    The set of generated graphs can be downloaded from (https://github.com/SOM-Research/metaScience-SoftwareConferences).

  11. 11.

    The set of results obtained are available at (https://github.com/SOM-Research/metaScience-SoftwareConferences).

  12. 12.

    Calculated as the average value of time executions of three consecutive executions in an Intel Core i7 machine, with 8 GB of RAM and Windows 7.

  13. 13.

    We obtained a \(R^2\) value of 0.243 (Nagelkerke index).

  14. 14.

    This is true even when we keep in mind that the way this variable is defined (i.e., all are considered newcomers in the first evaluated edition) makes it obvious to observe at least a slight decrease in the Newcomers value.

  15. 15.

    http://som-research.uoc.edu/tools/metaScience/.

References

  1. Biryukov, M., & Dong, C. (2010). Analysis of computer science communities based on DBLP. ECDL Conference, 6273, 228–235.

    Google Scholar 

  2. Blondel, V. D., Guillaume, J. L., Lambiotte, R., & Lefebvre, E. (2008). Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment, 10, 1000.

    MATH  Google Scholar 

  3. Dunn, O. (1961). Multiple comparisons among means. Journal of the American Statistical Association, 56(293), 52–64.

    MathSciNet  MATH  Article  Google Scholar 

  4. Elmacioglu, E., & Lee, D. (2005). On six degrees of separation in DBLP-DB and more. ACM SIGMOD Record, 34(2), 33.

    Article  Google Scholar 

  5. Franceschet, M. (2010). The role of conference publications in CS. Communications of the ACM, 53(12), 129.

    Article  Google Scholar 

  6. Franceschet, M. (2011). Collaboration in computer science: A network science approach. Journal of the American Society for Information Science and Technology, 62(10), 1992–2012.

    Article  Google Scholar 

  7. Freyne, J., Coyle, L., Smyth, B., & Cunningham, P. (2010). Relative status of journal and conference publications in computer science. Communications of the ACM, 53(11), 124.

    Article  Google Scholar 

  8. Hassan, A.E., & Holt, R.C. (2004). The Small World of Software Reverse Engineering

  9. Hollander, M., Douglas, W., & Chicken, E. (1999). Nonparametric statistical methods (2nd ed.). NewYork: Wiley.

    Google Scholar 

  10. Huang, J., Zhuang, Z., Li, J., & Giles, C. L. (2008). Collaboration over time: Characterizing and modeling network evolution. In WSDM Conference, pp. 107–116

  11. Konietschke, H., Hothorn, L., & Brunner, E. (2012). Rank-based multiple test procedures and simultaneous confidence intervals. Electronic Journal of Statistics, 6, 738–759.

    MathSciNet  MATH  Article  Google Scholar 

  12. McKerlich, R., Ives, C., & McGreal, R. (2012). The large-scale structure of journal citation networks. Journal of the American Society for Information Science and Technology, 63(4), 837–842.

    Article  Google Scholar 

  13. Newman, M. (2001). Scientific collaboration networks. I. Network construction and fundamental results. Physical Review, 64(1), 1–8.

    Google Scholar 

  14. Newman, M. E. J. (2000). Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. Physical Review, 64, 17.

    Google Scholar 

  15. Rahm, E. (2008). Comparing the scientific impact of conference and journal publications in computer science. Information Services and Use, 28, 127–128.

    Article  Google Scholar 

  16. Sarigöl, E., Pfitzner, R., Scholtes, I., Garas, A., & Schweitzer, F. (2014). Predicting scientific success based on coauthorship networks. EPJ Data Science, 3(1), 9.

    Article  Google Scholar 

  17. Tool, graphs and metric results used in the study (2016). https://github.com/SOM-Research/metaScience-SoftwareConferences.

  18. Vardi, M. Y. (2009). Conferences versus journals in computing research. Communications of the ACM, 52(5), 5.

    Article  Google Scholar 

  19. Vasilescu, B., Serebrenik, A., & Mens, T. (2013). A historical dataset of software engineering conferences. In MSR conference, pp. 373–376

  20. Vasilescu, B., Serebrenik, A., Mens, T., van den Brand, M. G., Pek, E., Van Den Brand, M. G. J., et al. (2014). How healthy are software engineering conferences. Science of Computer Programming, 89(Part C), 251–272.

    Article  Google Scholar 

  21. Veselin, G., Zhaochen, G., Serrano, D., Tansey, B., Barbosa, D., & Stroulia, E. (2009). An environment for building, exploring and querying academic social networks. In MEDES conference, p. 42

  22. Wasserman, S., & Hollander, M. (1994). Social network analysis, methods and applications. Cambridge: Cambridge University Press.

    Google Scholar 

  23. Wilcoxon, F. (1945). Individual Comparisons by Ranking Methods. Biom Bull, 1(6), 80–83.

    Article  Google Scholar 

  24. Yan, E., & Ding, Y. (2011). Discovering author impact: A PageRank perspective. Information Processing & Management, 47(1), 125–134.

    Article  Google Scholar 

  25. Yoshikane, F., & Kageura, K. (2004). Comparative analysis of coauthorship networks of different domains: The growth and change of networks. Scientometrics, 60(3), 433–444.

    Article  Google Scholar 

  26. Zimmerman, D., & Zumbo, B. (1992). Parametric alternatives to the student t-test under violation of normality and homogeneity of variance. Perceptual and Motor Skills, 74, 835–844.

    Article  Google Scholar 

Download references

Acknowledgments

We would like to thank Claudia Malpica for providing useful feedback on earlier versions of this paper.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Javier Luis Cánovas Izquierdo.

Appendix: List of selected CORE-ranked conferences

Appendix: List of selected CORE-ranked conferences

Conference name Rank Eds.
ACM Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication A* 31
ACM Conference on Computer and Communications Security A* 21
ACM Conference on Object Oriented Programming Systems Languages and Applications A* 28
ACM International Symposium on Computer Architecture A* 41
ACM Multimedia A* 22
ACM SIGSOFT International Symposium on the Foundations of Software Engineering A 21
ACM Symposium on Information, Computer and Communications Security B 8
ACM/IEEE International Conference on Distributed Smart Cameras B 8
ACM/IFIP/USENIX International Middleware Conference A 14
ACM-SIGACT Symposium on Principles of Programming Languages A* 41
Annual Computer Security Applications Conference A 25
Architectural Support for Programming Languages and Operating Systems A* 19
ASIAN Symposium on Programming Languages and Systems B 15
Asia-Pacific Software Engineering Conference B 21
Automated Software Engineering Conference A 18
Computational Intelligence in Security for Information Systems B 8
Conference on Agile Software Development B 10
Conference on RFID Security C 5
Conference on Security and Cryptography for Networks B 7
Conference on Software Engineering Education (and Training) C 26
Conference on the Quality of Software Architectures B 8
Dynamic Languages Symposium B 8
European Conference on Object-Oriented Programming A 27
European Conference on Pattern Languages of Programs B 12
European Conference on Software Architecture A 6
European Symposium on Programming A 23
European Symposium On Research In Computer Security A 19
Eurosys Conference A 9
Foundations of Aspect-Oriented Languages C 7
Foundations of Software Science and Computational Structures A 17
Fundamental Approaches to Software Engineering B 17
GI International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment C 11
IEEE Bioinformatics and Bioengineering C 14
IEEE Computer Security Foundations Symposium A 8
IEEE International Conference on Global Software Engineering C 9
IEEE International On-Line Testing Symposium C 12
IEEE International Requirements Engineering Conference A 18
IEEE International Symposium on High Assurance Systems Engineering B 15
IEEE International Symposium on Performance Analysis of Systems and Software C 14
IEEE Symposium on Field Programmable Custom Computing Machines A 20
IEEE Symposium on Rapid Prototyping C 10
IEEE Symposium on Security and Privacy A* 35
IEEE/IFIP International Conference on Dependable Systems A 15
IEEE/IFIP Working Conference on Software Architecture A 8
IFIP Joint International Conference on Formal Description Techniques and Protocol Specification, Testing, And Verification A 22
IFIP WG 11.3 Working Conference on Data and Applications Security A 28
Information Security C 14
Information Security Practice and Experience Conference B 10
International Computer Software and Applications Conference B 21
International Conference on Agile Software Development B 12
International Conference on Applied Cryptography and Network Security B 12
International Conference on Availability, Reliability and Security B 9
International Conference on Compiler Construction A 22
International Conference on Computer Safety, Reliability and Security B 17
International Conference on Coordination Models and Languages A 16
International Conference on Cryptology and Network Security B 10
International Conference on Dependable, Autonomic and Secure Computing C 6
International Conference on Evaluation of Novel Approaches to Software Engineering B 8
International Conference on Formal Engineering Methods B 16
International Conference on Functional Programming A* 19
International Conference on Generative Programming and Component Engineering B 13
International Conference on Information and Communications Security B 16
International Conference on Information Security and Assurance C 7
International Conference on Information Systems Security B 10
International Conference on Model Driven Engineering Languages and Systems B 10
International Conference on Model Transformation B 7
International Conference on network and System Security B 6
International Conference on Principles and Practice of Constraint Programming A 20
International Conference on Principles and Practice of Declarative Programming B 16
International Conference on Program Comprehension C 9
International Conference on Provable Security B 8
International Conference on Quality Software B 12
International Conference on Reliable Software Technologies B 24
International Conference on Risks and Security of Internet and Systems C 7
International Conference on Software Composition B 12
International Conference on Software Engineering A* 35
International Conference on Software Engineering and Formal Methods B 12
International Conference on Software Engineering and Knowledge Engineering B 22
International Conference on Software Language Engineering B 7
International Conference on Software Methods and Tools B 10
International Conference on Software Testing, Verification and Validation C 7
International Conference on Tests and Proofs B 8
International Conference on Trust, Privacy and Security in Digital Business B 11
International Conference on Virtual Execution Environments A 10
International Symposium Component-Based Software Engineering B 11
International Symposium on Automated Technology for Verification and Analysis A 11
International Symposium on Empirical Software Engineering and Measurement A 7
International Symposium on High Performance Computer Architecture A* 20
International Symposium on Memory Management A 12
International Symposium on Microarchitecture A 42
International Symposium on Software Reliability Engineering A 24
International Symposium on Software Testing and Analysis A 16
International Symposium on Theoretical Aspects of Software Engineering C 8
Joint Working Conference on Secure Information Networks: Communications and Multimedia Security C 12
Practical Aspects of Declarative Languages B 16
Prague Stringology Conference C 18
Principles and Practice of Programming in Java C 10
Product Focused Software Process Improvement B 14
Security and Privacy for Communication Networks A 9
Symposium On Usable Privacy and Security B 6
Tools and Algorithms for Construction and Analysis of Systems A 20
Usenix Symposium on Operating Systems Design and Implementation A* 10
Verification, Model Checking and Abstract Interpretation B 13

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Cánovas Izquierdo, J.L., Cosentino, V. & Cabot, J. Analysis of co-authorship graphs of CORE-ranked software conferences. Scientometrics 109, 1665–1693 (2016). https://doi.org/10.1007/s11192-016-2136-6

Download citation

Keywords

  • Co-authorship graph
  • DBLP dataset
  • CORE conference ranking
  • Scientometrics
  • Computer science
  • Software engineering