A Novel Approach to Evaluate Community Detection Algorithms on Ground Truth

  • Giulio RossettiEmail author
  • Luca Pappalardo
  • Salvatore Rinzivillo
Part of the Studies in Computational Intelligence book series (SCI, volume 644)


Evaluating a community detection algorithm is a complex task due to the lack of a shared and universally accepted definition of community. In literature, one of the most common way to assess the performances of a community detection algorithm is to compare its output with given ground truth communities by using computationally expensive metrics (i.e., Normalized Mutual Information). In this paper we propose a novel approach aimed at evaluating the adherence of a community partition to the ground truth: our methodology provides more information than the state-of-the-art ones and is fast to compute on large-scale networks. We evaluate its correctness by applying it to six popular community detection algorithms on four large-scale network datasets. Experimental results show how our approach allows to easily evaluate the obtained communities on the ground truth and to characterize the quality of community detection algorithms.


Ground Truth Community Detection Normalize Mutual Information Community Evaluation Community Detection Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was partially funded by the European Community’s H2020 Program under the funding scheme “FETPROACT-1-2014: Global Systems Science (GSS)”, grant agreement #641191 CIMPLEX “Bringing CItizens, Models and Data together in Participatory, Interactive SociaL EXploratories”, Our research is also supported by the European Community’s H2020 Program under the scheme “INFRAIA-1-2014-2015: Research Infrastructures”, grant agreement #654024 “SoBigData: Social Mining & Big Data Ecosystem”,


  1. 1.
    Fortunato, S.: Community detection in graphs, Physics Reports, vol. 486, no. 3–5, pp. 75–174 (2010).
  2. 2.
    Coscia, M., Giannotti, F., Pedreschi, D.: A classification for community discovery methods in complex networks. Stat. Anal. Data Min., 4(5), 512–546 (2011).
  3. 3.
    Lancichinetti, A., Fortunato, S., Radicchi, F.: Benchmark graphs for testing community detection algorithms. Phys. Rev. E, 78(4), 046110 (2008).
  4. 4.
    Bhat, S., Abulaish, M.: Overlapping social network communities and viral marketing. In: International Symposium on Computational and Business Intelligence, pp. 243–246, Aug 2013Google Scholar
  5. 5.
    Wu, X., Liu, Z.: How community structure influences epidemic spread in social networks. Phys. A: Stat. Mech. Appl. 387, 623–630 (2008)CrossRefGoogle Scholar
  6. 6.
    Rossetti, G., Pappalardo, L., Kikas, R., Pedreschi, D., Giannotti, F., Dumas, M.: Community-centric analysis of user engagement in skype social network. In: Proceedings of the 2015 ACM/IEEE International Conference on Advances in Social Network Analysis and Mining (2015)Google Scholar
  7. 7.
    Rossetti, G., Guidotti, R., Pennacchioli, D., Pedreschi, D., Giannotti, D.: Interaction prediction in dynamic networks exploiting community discovery. In: Proceedings of the 2015 ACM/IEEE International Conference on Advances in Social Network Analysis and Mining (2015)Google Scholar
  8. 8.
    Rinzivillo, S., Mainardi, S., Pezzoni, F., Coscia, M., Giannotti, F., Pedreschi, D.: Discovering the geographical borders of human mobility. KI - Künstliche Intelligenz (2012)Google Scholar
  9. 9.
    Bagrow, J.P., Lin, Y.-R.: Mesoscopic structure and social aspects of human mobility. PLoS ONE 7(5), p. e37676, 2012.
  10. 10.
    Lancichinetti, A., Fortunato, S.: Benchmarks for testing community detection algorithms on directed and weighted graphs with overlapping communities. Phys. Rev. E 80(1), 016118 (2009)CrossRefGoogle Scholar
  11. 11.
    McDaid, A.F., Greene, D., Hurley, N.J.: Normalized mutual information to evaluate overlapping community finding algorithms. CoRR, arXiv:1110.2515 (2011)
  12. 12.
    Detecting the overlapping and hierarchical community structure in complex networks. New J. Phys. (2009)Google Scholar
  13. 13.
    Blondel, V.D., Guillaume, J.-L., Lambiotte, R., Lefebvre, E.: Fast unfolding of communities in large networks. J. Stat. Mech.: Theory Exp. 2008(10), P10008 (2008)Google Scholar
  14. 14.
    Rosvall, M., Bergstrom, C.T.: Maps of random walks on complex networks reveal community structure. Proc. National Acad. Sci. 105(4), 1118–1123 (2008)CrossRefGoogle Scholar
  15. 15.
    Palla, G., Derényi, I., Farkas, I., Vicsek, T.: Uncovering the overlapping community structure of complex networks in nature and society. Nature 435(7043), 814–818 (2005)CrossRefGoogle Scholar
  16. 16.
    Coscia, M., Rossetti, G., Giannotti, F., Pedreschi, D.: Demon: a local-first discovery method for overlapping communities. In: Agarwal, D., Pei, J. (eds.), KDD, Q. Y. 0001, pp. 615–623. ACM (2012)Google Scholar
  17. 17.
    Cazabet, R., Amblard, F., Hanachi, C.: Detection of overlapping communities in dynamical social networks. In: SocialCom, pp. 309–314 (2010)Google Scholar
  18. 18.
    Jaewon, Y., Leskovec, J.: Defining and evaluating network communities based on ground-truth. Knowl. Inf, Syst (2015)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Giulio Rossetti
    • 1
    Email author
  • Luca Pappalardo
    • 1
  • Salvatore Rinzivillo
    • 1
  1. 1.Institute of Information Science and Technologies (ISTI)National Research Council of Italy (CNR)PisaItaly

Personalised recommendations