In most areas of computer science (CS), and in the software domain in particular, international conferences are as important as journals as a venue to disseminate research results. This has resulted in the creation of rankings to provide quality assessment of conferences (specially used for academic promotion purposes) like the well-known CORE ranking created by the Computing Research and Education Association of Australasia. In this paper we analyze 102 CORE-ranked conferences in the software area (covering all aspects of software engineering, programming languages, software architectures and the like) included in the DBLP dataset, an online reference for computers science bibliographic information. We define a suite of metrics focusing on the analysis of the co-authorship graph of the conferences, where authors are represented as nodes and co-authorship relationships as edges. Our aim is to first characterize the patterns and structure of the community of researchers in software conferences. We then try to see if these values depend on the quality rank of the conference justifying this way the existence of the different classifications in the CORE-ranking system.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Microsoft Academic Search Conference Ranking. http://bit.ly/1WMsg4y.
Simple H-Index Estimator. http://shine.icomp.ufam.edu.br/about.php.
Initiative sponsored by GII (Group of Italian Professors of Computer Engineering) and GRIN (Group of Italian Professors of Computer Science). http://valutazione.unibas.it/cs-conference-rating/conferenceRating.jsf.
A more detailed description about how the classification is defined can be found at http://www.core.edu.au/documents/RankingDescriptions2014.pdf.
Available at http://portal.core.edu.au/conf-ranks as CSV file.
Available at http://dblp.uni-trier.de/xml.
Provided by http://dblp.l3s.de.
The set of generated graphs can be downloaded from (https://github.com/SOM-Research/metaScience-SoftwareConferences).
The set of results obtained are available at (https://github.com/SOM-Research/metaScience-SoftwareConferences).
Calculated as the average value of time executions of three consecutive executions in an Intel Core i7 machine, with 8 GB of RAM and Windows 7.
We obtained a \(R^2\) value of 0.243 (Nagelkerke index).
This is true even when we keep in mind that the way this variable is defined (i.e., all are considered newcomers in the first evaluated edition) makes it obvious to observe at least a slight decrease in the Newcomers value.
Biryukov, M., & Dong, C. (2010). Analysis of computer science communities based on DBLP. ECDL Conference, 6273, 228–235.
Blondel, V. D., Guillaume, J. L., Lambiotte, R., & Lefebvre, E. (2008). Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment, 10, 1000.
Dunn, O. (1961). Multiple comparisons among means. Journal of the American Statistical Association, 56(293), 52–64.
Elmacioglu, E., & Lee, D. (2005). On six degrees of separation in DBLP-DB and more. ACM SIGMOD Record, 34(2), 33.
Franceschet, M. (2010). The role of conference publications in CS. Communications of the ACM, 53(12), 129.
Franceschet, M. (2011). Collaboration in computer science: A network science approach. Journal of the American Society for Information Science and Technology, 62(10), 1992–2012.
Freyne, J., Coyle, L., Smyth, B., & Cunningham, P. (2010). Relative status of journal and conference publications in computer science. Communications of the ACM, 53(11), 124.
Hassan, A.E., & Holt, R.C. (2004). The Small World of Software Reverse Engineering
Hollander, M., Douglas, W., & Chicken, E. (1999). Nonparametric statistical methods (2nd ed.). NewYork: Wiley.
Huang, J., Zhuang, Z., Li, J., & Giles, C. L. (2008). Collaboration over time: Characterizing and modeling network evolution. In WSDM Conference, pp. 107–116
Konietschke, H., Hothorn, L., & Brunner, E. (2012). Rank-based multiple test procedures and simultaneous confidence intervals. Electronic Journal of Statistics, 6, 738–759.
McKerlich, R., Ives, C., & McGreal, R. (2012). The large-scale structure of journal citation networks. Journal of the American Society for Information Science and Technology, 63(4), 837–842.
Newman, M. (2001). Scientific collaboration networks. I. Network construction and fundamental results. Physical Review, 64(1), 1–8.
Newman, M. E. J. (2000). Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. Physical Review, 64, 17.
Rahm, E. (2008). Comparing the scientific impact of conference and journal publications in computer science. Information Services and Use, 28, 127–128.
Sarigöl, E., Pfitzner, R., Scholtes, I., Garas, A., & Schweitzer, F. (2014). Predicting scientific success based on coauthorship networks. EPJ Data Science, 3(1), 9.
Tool, graphs and metric results used in the study (2016). https://github.com/SOM-Research/metaScience-SoftwareConferences.
Vardi, M. Y. (2009). Conferences versus journals in computing research. Communications of the ACM, 52(5), 5.
Vasilescu, B., Serebrenik, A., & Mens, T. (2013). A historical dataset of software engineering conferences. In MSR conference, pp. 373–376
Vasilescu, B., Serebrenik, A., Mens, T., van den Brand, M. G., Pek, E., Van Den Brand, M. G. J., et al. (2014). How healthy are software engineering conferences. Science of Computer Programming, 89(Part C), 251–272.
Veselin, G., Zhaochen, G., Serrano, D., Tansey, B., Barbosa, D., & Stroulia, E. (2009). An environment for building, exploring and querying academic social networks. In MEDES conference, p. 42
Wasserman, S., & Hollander, M. (1994). Social network analysis, methods and applications. Cambridge: Cambridge University Press.
Wilcoxon, F. (1945). Individual Comparisons by Ranking Methods. Biom Bull, 1(6), 80–83.
Yan, E., & Ding, Y. (2011). Discovering author impact: A PageRank perspective. Information Processing & Management, 47(1), 125–134.
Yoshikane, F., & Kageura, K. (2004). Comparative analysis of coauthorship networks of different domains: The growth and change of networks. Scientometrics, 60(3), 433–444.
Zimmerman, D., & Zumbo, B. (1992). Parametric alternatives to the student t-test under violation of normality and homogeneity of variance. Perceptual and Motor Skills, 74, 835–844.
We would like to thank Claudia Malpica for providing useful feedback on earlier versions of this paper.
Appendix: List of selected CORE-ranked conferences
Appendix: List of selected CORE-ranked conferences
|ACM Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication||A*||31|
|ACM Conference on Computer and Communications Security||A*||21|
|ACM Conference on Object Oriented Programming Systems Languages and Applications||A*||28|
|ACM International Symposium on Computer Architecture||A*||41|
|ACM SIGSOFT International Symposium on the Foundations of Software Engineering||A||21|
|ACM Symposium on Information, Computer and Communications Security||B||8|
|ACM/IEEE International Conference on Distributed Smart Cameras||B||8|
|ACM/IFIP/USENIX International Middleware Conference||A||14|
|ACM-SIGACT Symposium on Principles of Programming Languages||A*||41|
|Annual Computer Security Applications Conference||A||25|
|Architectural Support for Programming Languages and Operating Systems||A*||19|
|ASIAN Symposium on Programming Languages and Systems||B||15|
|Asia-Pacific Software Engineering Conference||B||21|
|Automated Software Engineering Conference||A||18|
|Computational Intelligence in Security for Information Systems||B||8|
|Conference on Agile Software Development||B||10|
|Conference on RFID Security||C||5|
|Conference on Security and Cryptography for Networks||B||7|
|Conference on Software Engineering Education (and Training)||C||26|
|Conference on the Quality of Software Architectures||B||8|
|Dynamic Languages Symposium||B||8|
|European Conference on Object-Oriented Programming||A||27|
|European Conference on Pattern Languages of Programs||B||12|
|European Conference on Software Architecture||A||6|
|European Symposium on Programming||A||23|
|European Symposium On Research In Computer Security||A||19|
|Foundations of Aspect-Oriented Languages||C||7|
|Foundations of Software Science and Computational Structures||A||17|
|Fundamental Approaches to Software Engineering||B||17|
|GI International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment||C||11|
|IEEE Bioinformatics and Bioengineering||C||14|
|IEEE Computer Security Foundations Symposium||A||8|
|IEEE International Conference on Global Software Engineering||C||9|
|IEEE International On-Line Testing Symposium||C||12|
|IEEE International Requirements Engineering Conference||A||18|
|IEEE International Symposium on High Assurance Systems Engineering||B||15|
|IEEE International Symposium on Performance Analysis of Systems and Software||C||14|
|IEEE Symposium on Field Programmable Custom Computing Machines||A||20|
|IEEE Symposium on Rapid Prototyping||C||10|
|IEEE Symposium on Security and Privacy||A*||35|
|IEEE/IFIP International Conference on Dependable Systems||A||15|
|IEEE/IFIP Working Conference on Software Architecture||A||8|
|IFIP Joint International Conference on Formal Description Techniques and Protocol Specification, Testing, And Verification||A||22|
|IFIP WG 11.3 Working Conference on Data and Applications Security||A||28|
|Information Security Practice and Experience Conference||B||10|
|International Computer Software and Applications Conference||B||21|
|International Conference on Agile Software Development||B||12|
|International Conference on Applied Cryptography and Network Security||B||12|
|International Conference on Availability, Reliability and Security||B||9|
|International Conference on Compiler Construction||A||22|
|International Conference on Computer Safety, Reliability and Security||B||17|
|International Conference on Coordination Models and Languages||A||16|
|International Conference on Cryptology and Network Security||B||10|
|International Conference on Dependable, Autonomic and Secure Computing||C||6|
|International Conference on Evaluation of Novel Approaches to Software Engineering||B||8|
|International Conference on Formal Engineering Methods||B||16|
|International Conference on Functional Programming||A*||19|
|International Conference on Generative Programming and Component Engineering||B||13|
|International Conference on Information and Communications Security||B||16|
|International Conference on Information Security and Assurance||C||7|
|International Conference on Information Systems Security||B||10|
|International Conference on Model Driven Engineering Languages and Systems||B||10|
|International Conference on Model Transformation||B||7|
|International Conference on network and System Security||B||6|
|International Conference on Principles and Practice of Constraint Programming||A||20|
|International Conference on Principles and Practice of Declarative Programming||B||16|
|International Conference on Program Comprehension||C||9|
|International Conference on Provable Security||B||8|
|International Conference on Quality Software||B||12|
|International Conference on Reliable Software Technologies||B||24|
|International Conference on Risks and Security of Internet and Systems||C||7|
|International Conference on Software Composition||B||12|
|International Conference on Software Engineering||A*||35|
|International Conference on Software Engineering and Formal Methods||B||12|
|International Conference on Software Engineering and Knowledge Engineering||B||22|
|International Conference on Software Language Engineering||B||7|
|International Conference on Software Methods and Tools||B||10|
|International Conference on Software Testing, Verification and Validation||C||7|
|International Conference on Tests and Proofs||B||8|
|International Conference on Trust, Privacy and Security in Digital Business||B||11|
|International Conference on Virtual Execution Environments||A||10|
|International Symposium Component-Based Software Engineering||B||11|
|International Symposium on Automated Technology for Verification and Analysis||A||11|
|International Symposium on Empirical Software Engineering and Measurement||A||7|
|International Symposium on High Performance Computer Architecture||A*||20|
|International Symposium on Memory Management||A||12|
|International Symposium on Microarchitecture||A||42|
|International Symposium on Software Reliability Engineering||A||24|
|International Symposium on Software Testing and Analysis||A||16|
|International Symposium on Theoretical Aspects of Software Engineering||C||8|
|Joint Working Conference on Secure Information Networks: Communications and Multimedia Security||C||12|
|Practical Aspects of Declarative Languages||B||16|
|Prague Stringology Conference||C||18|
|Principles and Practice of Programming in Java||C||10|
|Product Focused Software Process Improvement||B||14|
|Security and Privacy for Communication Networks||A||9|
|Symposium On Usable Privacy and Security||B||6|
|Tools and Algorithms for Construction and Analysis of Systems||A||20|
|Usenix Symposium on Operating Systems Design and Implementation||A*||10|
|Verification, Model Checking and Abstract Interpretation||B||13|
About this article
Cite this article
Cánovas Izquierdo, J.L., Cosentino, V. & Cabot, J. Analysis of co-authorship graphs of CORE-ranked software conferences. Scientometrics 109, 1665–1693 (2016). https://doi.org/10.1007/s11192-016-2136-6
- Co-authorship graph
- DBLP dataset
- CORE conference ranking
- Computer science
- Software engineering