Graph-Based Multimodal Clustering for Social Event Detection in Large Collections of Images

  • Georgios Petkos
  • Symeon Papadopoulos
  • Emmanouil Schinas
  • Yiannis Kompatsiaris
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8325)


A common approach to the problem of SED in collections of multimedia relies on the use of clustering methods. Due to the heterogeneity of features associated with multimedia items in such collections, such a clustering task is very challenging and special multimodal clustering approaches need to be deployed. In this paper, we present a scalable graph-based multimodal clustering approach for SED in large collections of multimedia. The proposed approach utilizes example relevant clusterings to learn a model of the “same event” relationship between two items in the multimodal domain and subsequently to organize the items in a graph. Two variants of the approach are presented: the first based on a batch and the second on an incremental community detection algorithm. Experimental results indicate that both variants provide excellent clustering performance.


Social media Social event detection Multimodal clustering 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-Up Robust Features (SURF). Comp. Vis. Image Underst. 110(3), 346–359 (2008)CrossRefGoogle Scholar
  2. 2.
    Becker, H., Naaman, M., Gravano, L.: Learning similarity metrics for event identification in social media. In: Proceedings of the Third ACM International Conference on Web Search and Data Mining, WSDM 2010, pp. 291–300. ACM, New York (2010)Google Scholar
  3. 3.
    Bekkerman, R., Jeon, J.: Multi-modal clustering for multimedia collections. In: CVPR (2007)Google Scholar
  4. 4.
    Brenner, M., Izquierdo, E.: Mediaeval benchmark: Social Event Detection in collaborative photo collections. In: MediaEval. CEUR Workshop Proceedings (2011)Google Scholar
  5. 5.
    Cai, X., Nie, F., Huang, H., Kamangar, F.: Heterogeneous image feature integration via multi-modal spectral clustering. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 1977–1984 (June 2011)Google Scholar
  6. 6.
    Fortunato, S.: Community detection in graphs. Physics Reports 486(3-5), 75–174 (2010)CrossRefMathSciNetGoogle Scholar
  7. 7.
    Goder, A., Filkov, V.: Consensus clustering algorithms: Comparison and refinement. In: Ian Munro, J. (ed.) Proceedings of the Workshop on Algorithm Engineering and Experiments, ALENEX 2008, San Francisco, California, USA, pp. 109–117. SIAM (January 19, 2008)Google Scholar
  8. 8.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: an update. SIGKDD Explorations Newsletter 11(1), 10–18 (2009)CrossRefGoogle Scholar
  9. 9.
    Jegou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(1), 117–128 (2011)CrossRefGoogle Scholar
  10. 10.
    Jégou, H., Douze, M., Schmid, C., Pérez, P.: Aggregating local descriptors into a compact image representation. In: 23rd IEEE Conference on Computer Vision & Pattern Recognition, CVPR 2010, pp. 3304–3311. IEEE Computer Society, San Francisco (2010)CrossRefGoogle Scholar
  11. 11.
    Khalidov, V., Forbes, F., Horaud, R.P.: Conjugate mixture models for clustering multimodal data. Neural Computation 23(2), 517–557 (2011)CrossRefzbMATHMathSciNetGoogle Scholar
  12. 12.
    Li, Y., Crandall, D.J., Huttenlocher, D.P.: Landmark classification in large-scale image collections. In: IEEE 12th International Conference on Computer Vision, ICCV 2009, Kyoto, Japan, September 27 - October 4, pp. 1957–1964. IEEE (2009)Google Scholar
  13. 13.
    Liu, X., Troncy, R., Huet, B.: Using social media to identify events. In: ACM Multimedia 3rd Workshop on Social Media, WSM 2011, Scottsdale, Arizona, USA, November 18-December 1, p. 11 (2011)Google Scholar
  14. 14.
    Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press, New York (2008)CrossRefzbMATHGoogle Scholar
  15. 15.
    Nguyen, N.P., Dinh, T.N., Xuan, Y., Thai, M.T.: Adaptive algorithms for detecting community structure in dynamic social networks. In: 30th IEEE International Conference on Computer Communications, Joint Conference of the IEEE Computer and Communications Societies, INFOCOM 2011, Shanghai, China, April 10-15, pp. 2282–2290. IEEE (2011)Google Scholar
  16. 16.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. Int. J. Comput. Vision 42(3), 145–175 (2001)CrossRefzbMATHGoogle Scholar
  17. 17.
    Papadopoulos, S., Kompatsiaris, Y., Vakali, A., Spyridonos, P.: Community detection in social media. Data Mining and Knowledge Discovery 24(3), 515–554 (2012)CrossRefGoogle Scholar
  18. 18.
    Papadopoulos, S., Schinas, E., Mezaris, V., Troncy, R., Kompatsiaris, I.: The 2012 Social Event Detection dataset. In: 4th ACM Multimedia Systems, Dataset Session, MMSys 2013, Oslo, Norway, February 27-March 1 (2013)Google Scholar
  19. 19.
    Papadopoulos, S., Schinas, E., Mezaris, V., Troncy, R., Kompatsiaris, Y.: Social Event Detection at MediaEval 2012: Challenges, Dataset and Evaluation. In: MediaEval 2012 Workshop, Pisa, Italy, October 4-5 (2012)Google Scholar
  20. 20.
    Papadopoulos, S., Troncy, R., Mezaris, V., Huet, B., Kompatsiaris, I.: Social Event Detection at Mediaeval 2011: Challenges, dataset and evaluation. In: MediaEval. CEUR Workshop Proceedings (2011)Google Scholar
  21. 21.
    Papadopoulos, S., Zigkolis, C., Kompatsiaris, Y., Vakali, A.: CERTH@Mediaeval 2011 social event detection task. In: MediaEval. CEUR Workshop Proceedings (2011)Google Scholar
  22. 22.
    Papadopoulos, S., Zigkolis, C., Kompatsiaris, Y., Vakali, A.: Cluster-based landmark and event detection for tagged photo collections. IEEE Multimedia 18(1), 52–63 (2011)CrossRefGoogle Scholar
  23. 23.
    Petkos, G., Papadopoulos, S., Kompatsiaris, Y.: Social event detection using multimodal clustering and integrating supervisory signals. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, ICMR 2012, pp. 23:1–23:8. ACM, New York (2012)Google Scholar
  24. 24.
    Phuvipadawat, S., Murata, T.: Breaking news detection and tracking in twitter. In: IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, vol. 3, pp. 120–123 (2010)Google Scholar
  25. 25.
    Rendle, S., Schmidt-Thieme, L.: Scaling record linkage to non-uniform distributed class sizes. In: Washio, T., Suzuki, E., Ting, K.M., Inokuchi, A. (eds.) PAKDD 2008. LNCS (LNAI), vol. 5012, pp. 308–319. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  26. 26.
    Reuter, T., Cimiano, P.: Event-based classification of social media streams. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, ICMR 2012, pp. 22:1–22:8. ACM, New York (2012)Google Scholar
  27. 27.
    Schinas, E., Mantziou, E., Papadopoulos, S., Petkos, G., Kompatsiaris, Y.: CERTH @ Mediaeval 2013 Social Event Detection Task. In: MediaEval. CEUR Workshop Proceedings (2013)Google Scholar
  28. 28.
    Schinas, E., Petkos, G., Papadopoulos, S., Kompatsiaris, Y.: CERTH @ Mediaeval 2012 Social Event Detection Task. In: MediaEval. CEUR Workshop Proceedings, vol. 927 (2012)Google Scholar
  29. 29.
    Snoek, C.G.M., Worring, M., Smeulders, A.W.M.: Early versus late fusion in semantic video analysis. In: Proceedings of the 13th Annual ACM International Conference on Multimedia, MULTIMEDIA 2005, pp. 399–402. ACM, New York (2005)Google Scholar
  30. 30.
    Xu, X., Yuruk, N., Feng, Z., Schweiger, T.A.J.: Scan: a structural clustering algorithm for networks. In: Proceedings of the 13th ACM SIGKDD, KDD 2007, pp. 824–833. ACM, NY (2007)Google Scholar
  31. 31.
    Ye, Z., Hu, S., Yu, J.: Adaptive clustering algorithm for community detection in complex networks. Physical Review E 78(4) (2008)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Georgios Petkos
    • 1
  • Symeon Papadopoulos
    • 1
  • Emmanouil Schinas
    • 1
  • Yiannis Kompatsiaris
    • 1
  1. 1.Information Technologies Institute, Centre for Research and Technology HellasGreece

Personalised recommendations