Not all areas are equal: analysis of citations in information security research

  • Steffen WendzelEmail author
  • Cédric Lévy-Bencheton
  • Luca Caviglione


The understanding of the inner workings of a research community is essential to evaluate the impact of an author as well as to decide where and how to publish results. One of the key metrics is the number of citations that a publication receives. In parallel, information security is now a key and strategic area, partially fueled by the advent of the Internet of Things (IoT) and the need of pursuing cybercriminals by using digital forensics techniques. Therefore, this paper analyzes several factors influencing the number of citations in the domain of information security, such as differences between journal and conference publications, or the impact of the number of pages and the length of the abstract. To obtain quantitative results, we investigated papers of six sub-disciplines, i.e., anonymity and privacy, cryptography, information hiding, IoT and Cyber-Physical System security, digital forensics and incident response, and network security. For each sub-domain, we used metadata of 5000 publications collected from IEEE-Xplore. Results indicate some clear behaviors, for instance, papers tend to receive more citations when their abstract is longer and the number of references positively influences the performance of the work.


Scientometrics Science of security Bibliometrics Academic literature Information security Cyber security 


  1. Anderson, T. (2009). Conference reviewing considered harmful. ACM SIGOPS Operating Systems Review, 43(2), 108–116. Scholar
  2. Baldwin, J., Alhawi, O. M., Shaughnessy, S., Akinbi, A., & Dehghantanha, A. (2018). Emerging from the cloud: A bibliometric analysis of cloud forensics studies. In A. Dehghantanha, M. Conti, & T. Dargahi (Eds.), Cyber threat intelligence (pp. 311–331). Cham: Springer.CrossRefGoogle Scholar
  3. Bornmann, L., & Leydesdorff, L. (2018). Count highly-cited papers instead of papers with h citations: Use normalized citation counts and compare “like with like”!. Scientometrics, 115(2), 1119–1123. Scholar
  4. Burcham, M., Al-Zyoud, M., Carver, J. C., Alsaleh, M., Du, H., Gilani, F., Jiang, J., Rahman, A., Kafalı, O., Al-Shaer, E., & Williams, L. (2017). Characterizing scientific reporting in security literature: An analysis of ACM CCS and IEEE S&P papers. In Proceedings of the hot topics in science of security: Symposium and bootcamp, HoTSoS (pp. 13–23). ACM, New York.
  5. Cabaj, K., Caviglione, L., Mazurczyk, W., Wendzel, S., Woodward, A., & Zander, S. (2018). The new threats of information hiding: the road ahead. IT Professional, 20(3), 31–39.CrossRefGoogle Scholar
  6. Carver, J. C., Burcham, M., Kocak, S. A., Bener, A., Felderer, M., Gander, M., King, J., Markkula, J., Oivo, M., Sauerwein, C., & Williams, L. (2016). Establishing a baseline for measuring advancement in the science of security: An analysis of the 2015 IEEE security and privacy proceedings. In Proceedings of the symposium and bootcamp on the science of security, HotSos’16 (pp. 38–51). ACM, New York.
  7. Cavelty, M. D. (2018). Cybersecurity research meets science and technology studies. Politics and Governance, 6(2), 22–30.CrossRefGoogle Scholar
  8. Chakraborty, J., Pradhan, D., Dutta, H. S., Nandi, S., & Chakraborty, T. (2018). On good and bad intentions behind anomalous citation patterns among journals in computer sciences. CoRR arXiv:1807.10804
  9. Chakraborty, T. (2018). Role of interdisciplinarity in computer sciences: Quantification, impact and life trajectory. Scientometrics, 114(3), 1011–1029. Scholar
  10. Cobo, M., Jürgens, B., Herrero-Solana, V., Martínez, M., & Herrera-Viedma, E. (2018). Industry 4.0: A perspective based on bibliometric analysis. Procedia Computer Science, 139, 364–371. (6th International Conference on Information Technology and Quantitative Management).CrossRefGoogle Scholar
  11. Collberg, C., & Proebsting, T. A. (2016). Repeatability in computer systems research. Communications of the ACM, 59(3), 62–69. Scholar
  12. Corporation, M. (2010). The science of cyber security. Tech. Rep. JSR-10-102, MITRE.
  13. Fernandes, J. M., & Monteiro, M. P. (2017). Evolution in the number of authors of computer science publications. Scientometrics, 110, 529–539.CrossRefGoogle Scholar
  14. Fiala, D., & Tutoky, G. (2017). Computer science papers in web of science: A bibliometric analysis. Publications, 5(4), 23. Scholar
  15. Goodrum, A. A., McCain, K. W., Lawrence, S., & Giles, C. L. (2001). Scholarly publishing in the internet age: A citation analysis of computer science literature. Information Processing & Management, 37(5), 661–675.CrossRefGoogle Scholar
  16. Herley, C., & van Oorschot, P. C. (2017). Sok: Science, security and the elusive goal of security as a scientific pursuit. In 2017 IEEE symposium on security and privacy (SP) (pp. 99–120). IEEE.
  17. Humayed, A., Lin, J., Li, F., & Luo, B. (2017). Cyber-physical systems security-a survey. IEEE Internet of Things Journal, 4(6), 1802–1831.CrossRefGoogle Scholar
  18. Iqbal, W., Qadir, J., Tyson, G., Mian, A. N., Hassan, Su, & Crowcroft, J. (2019). A bibliometric analysis of publications in computer networking research. Scientometrics, 119(2), 1121–1155. Scholar
  19. Isenberg, P., Isenberg, T., Sedlmair, M., Chen, J., & Möller, T. (2017). Visualization as seen through its research paper keywords. IEEE Transactions on Visualization and Computer Graphics, 23(1), 771–780. Scholar
  20. Jia, H., & Saule, E. (2017). An analysis of citation recommender systems: Beyond the obvious. In Proceedings of the 2017 IEEE/ACM international conference on advances in social networks analysis and mining 2017, ASONAM ’17 (pp. 216–223). ACM, New York, NY.
  21. Kott, A. (2014). Towards fundamental science of cyber security. In R. E. Pino (Ed.), Network science and cybersecurity (pp. 1–13). New York: Springer.Google Scholar
  22. Kousha, K., & Thelwall, M. (2007). Google scholar citations and google web/url citations: A multi-discipline exploratory analysis. Journal of the Association for Information Science and Technology, 58(7), 1055–1065.Google Scholar
  23. Li, M. (2018). Classifying and ranking topic terms based on a novel approach: Role differentiation of author keywords. Scientometrics, 116(1), 77–100.CrossRefGoogle Scholar
  24. MacDonald, K. I., & Dressler, V. (2018). Using citation analysis to identify research fronts: A case study with the internet of things. Science & Technology Libraries, 37(2), 171–186. Scholar
  25. Makawana, P. R., & Jhaveri, R. H. (2018). A bibliometric analysis of recent research on machine learning for cyber security. In Y. C. Hu, S. Tiwari, K. K. Mishra, & M. C. Trivedi (Eds.), Intelligent Communication and Computational Technologies (pp. 213–226). Singapore: Springer Singapore.CrossRefGoogle Scholar
  26. Mazurczyk, W., & Caviglione, L. (2015). Information hiding as a challenge for malware detection. IEEE Security Privacy, 13(2), 89–93.CrossRefGoogle Scholar
  27. Meyer, B. (2011). The nastiness problem in computer science. Posting in the blog of Comm. ACM.
  28. Mogul, J. C. (2013). Towards more constructive reviewing of CS papers. ACM SIGCOMM Computer Communication Review, 43(3), 90–94. Scholar
  29. Muhamedyev, R. I., Aliguliyev, R. M., Shokishalov, Z. M., & Mustakayev, R. R. (2018). New bibliometric indicators for prospectivity estimation of research fields. Annals of Library and Information Studies, 65, 62–69.Google Scholar
  30. Shen, S., Rousseau, R., & Wang, D. (2018). Do papers with an institutional e-mail address receive more citations than those with a non-institutional one? Scientometrics, 115, 1039–1050.CrossRefGoogle Scholar
  31. Stanton, J., Mastrangelo, P., Stam, K., & Jolton, J. (2004). Behavioral information security: Two end user survey studies of motivation and security practices. In AMCIS 2004 proceedings (pp. 175).Google Scholar
  32. Sun, X., Ding, K., & Lin, Y. (2016). Mapping the evolution of scientific fields based on cross-field authors. Journal of Informetrics, 10(3), 750–761.CrossRefGoogle Scholar
  33. Vieira, E. S., & Gomes, J. A. (2009). Citations to scientific articles: Its distribution and dependence on the article features. Journal of Informetrics, 4, 1–13.CrossRefGoogle Scholar
  34. Wendzel, S. (2018). Get me cited, Scotty! Analysis of academic publications in covert channel/steganography research. In Proceedings of the 13th international conference on availability, reliability and security (ARES’18) (pp. 13:1–13:8). ACM, New York, NY.
  35. Yan, B. N., Lee, T. S., & Lee, T. P. (2015). Mapping the intellectual structure of the internet of things (IoT) field (2000–2014): a co-word analysis. Scientometrics, 105(2), 1285–1300.CrossRefGoogle Scholar
  36. Yang, K., & Meho, L. I. (2006). Citation analysis: A comparison of google scholar, scopus, and web of science. Proceedings of the Association for Information Science and Technology, 43(1), 1–15.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2019

Authors and Affiliations

  1. 1.Worms University of Applied SciencesWormsGermany
  2. 2.Fraunhofer FKIEBonnGermany
  3. 3.CetomeLondonUK
  4. 4.Institute for Applied Mathematics and Information TechnologiesGenoaItaly

Personalised recommendations